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Preface 



This Sourcebook is based upon presentations given and material 
used in a five-day Institute on Evaluation Strategies and Techniques 
for Public Library Children's Services funded by the United States 
Department of Education and sponsored by the School of Library and 
Information Studies of the University of Wisconsin-Madison in May 
1989. Fifty participants from twenty-five states were selected from over 
125 applicants to attend. A mixture of professional interests were 
represented including coordinators of children's departments in large 
metropolitan public libraries, consultants in children's services for 
public library systems and for state library agencies as well as educators 
of future children's librarians. 

The Sourcebook is intended for use by anyone with responsibility 
for providing beginning or continuing education for children's services 
librarians. It articulates, within the constraints of a single volume, the 
evaluation concerns that are pertinent to the community of those who 
serve children through public libraries as we move into the last decade 
of the twentieth century. 
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I. Planning and Evaluation 



Introduction 

According to one prominent evaluator, the purpose of evaluation is not to prove, but 
to improve. Often when we think of evaluation, what comes to mind is some final 
judgment of worth or of making a good case so that your prog"" sumves. But 
these purposes of evaluation are seen as missing the mark. Evaluation, as we re see,ng 
it for this Sourcebook, is an assessment of wo* inprogress, a means ofmfommg 
decisions. Evaluation asks the «,««««.«, "Are we there yetV and we're always ,n the 
process of arriving. 

in this Sourcebook, we are presenting a variety of evaluation approaches in the 
context of public library cKMren's services. In this chapter, the articles deal wrth the 
role of evaluation in the planning process, the importance of »«»S clear «!«.«< the 
target that is sought, and the gathering of information about the contexts in which our 
services occur. 

This Sourcebook advocates minimalist planning-and continual planning, Our 
definition of planning is "a series of successive approximations to a moving target." 
The implication of a series is that there is never a final plan or evaluation. Planning 
and evaluation to be useful in improvement of services must be a regular and repeated 
activity. The corollary to that conclusion is that planning and evaluation must be 
inexpensivc-not elaborate-or libraries cannot afford to carry it out repeatedly. The 
evaluation approaches included in this Sourcebook are ones that can be carried out 
with little effort or expenditure. 
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This section is titled "planning and evaluation," and it may be well to think of 
them as one word: "planningandevaluation" because it doesn't seem sensible to think 
of planning without evaluation or of evaluation in the al .vce of planning. Planning is 
the process of selecting targets for achievement; evaluation vs the process of assessing 
how close we are to achieving the target, 

This Introduction will quickly point to the planning and role setting process 
recently published by the Public Library Association (PLA) and at evaluation's place 
in it. V/e can make a number of observations about this process. 



7. Review and 
recycling 



1. Preparing 
to plan 
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2. Looking 
around 



6. The planning 
product 




THE 
PLANNING 
CYCLE 



3. Roles and 
mission 



5. Taking 
action 



4. Writing goals 
and objectives 



First, it is cyclical. If the process is used as an on-going series of repetitions, then it 
doesn't matter where the library starts in the process, the library will eventually 
complete all seven steps, and the point at which it began will be inconsequential. A 
more pertinent point is that evaluation can and should occur at every step of the cycle. 
When Preparing to Plan, the staffs capabilities at planing, the previous experience 
with planning, and the constraints that may limit planning decisions can all be 
assessed. Looking Around consists of evaluating the context in which the library is 
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operating and ike Romance of the library, to produce findings that mil gmde 
subsequent planning steps. Developing Holes and Mission involves woghng the 
various potential roles the library might adopt to select a limited -mmfcr to pursue. 

WH«„ g Goals and Oofectoes tata selecting ike best targets from among 
the many possible. Taking Action requires two kinds of evaluations: assessing 
alternative courses of action before settling on one to carry out and comparing the 
progress of the selected course of action with the expected schedule or ,ntermedu,te 
Zgel The Review and Recycling step is the one we would generally assocuue w,,h 
Jaluation-the step where achievements are compared with intentions, and 
decisions are made about continuing «**« programs or substituting others. 

So there are many micro-opportunities for evaluation within the macro planning 
process: surveys can be used to assess staff attitudes toward planning or atoen 
Zlness of services: output measures can be used to identify problems » performance 
or to check on progress. 

The Roles and Mission step deserves some particular attention. Planning involves 
making choices-choices to carry some things out and choices, implicitly or explicitly, 
notto carry out other things. As Peter Drucker has observed, setting pnonttes is easy, 
deciding on posteriorities is tough.* To aid public libraries in this necessary step, the 
Public library Association has identified eight role options for libraries. 

PUBLIC LIBRARY ROLES 

COMMUNITY ACTIVITIES CENTER 

COMMUNITY INFORMATION CENTER 

FORMAL EDUCATION SUPPORT CENTER 

INDEPENDENT LEARNING CENTER 

POPULAR MATERIALS LIBRARY 

PRESCHOOLER'S DOOR TO LEARNING 

REFERENCE LIBRARY 

RESEARCH CENTER 

• Martin, Lowell. The Public Library: Middle^ Crisis or Old Age Crisis/ Library Journal 108 
(January 1, 1983): 17-2* 
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Note that these roles are not age specific. The intention was that youth services 
could pursue any of these roles. If the library decided to emphasize Popular Material's 
Library and Reference Library as primary roles, youth services could participate in 
those roles along with services to adults. The only age-related role is Preschooler's 
Door to Learning, and this was intended to reflect the particular quality of the service 
rather than the age of the clientele. Unforturately, some have seen that role as the 
only place for youth services, and some have even renamed the role as Children's Door 
to Learning sc that all of children's services could be included in it. The effect of this 
change is to once again segregate youth services as separate from the rest of the 
library's program of services— the very effect the Public Library Association wanted 
to avoid. A suggestion }ias been made that this role should be called Door to Learning 
and should include adult literacy services as well as services to preschoolers, and the 
next edition may well reflect that change. 

To return to the Roles and Mission step: PIA recommends that a library choose one 
or two of the roles as primary and one or two as secondary, the remaining rolts to be 
performed at a maintenance level and to consume, together, no more than 20% of the 
library's effort. The purpose of this role selection is for the library "to concentrate and 
strengthen," as Lowell Martin put it— so the planning process acts like a funnel: wide 
open initially when the looking around is taking place and narrowed by the roles 
choice so that the choice of goals and objectives is within the areas of priority. 

It is helpful to see the roles, goals, and objectives as kinds of statements that have 
different functions. A role is a description of a service profile that gives an image of 
the kind of library the community will have. A goal is a statement of an ideal for the 
library, an end toward which the library intends to move, the Man from La Mancha's 
"unreachable star." An objective states how far toward the ideal end the library 
intends to move during the period being planned. While goals are by definition 
unattainable, objectives are attainable, measureable, and dated (so one knows when 
they should be attained.) The objective needs to include a measure of 
attainment— how far the library has progressed toward its goal. 
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The following scheme illustrates how these parts flow from one to the next. 

hole Who we are 

[example: COMMUNITY ACTIVITIES CENTER] 

GOAL What direction we're heading 

[example: TO INCREASE USE OF THE LIBRARY] 

nrnnmvE How far we're going this time 
OBJECHVE [example: TO INCREASE LIBRARY VISITS PER 
CAPITA FROM 3.5 TO 4.5 BY JUNE !, 1990] 



OUTPUT How we'll tell whether we're there 

MEASURE [example: LIBRARY VISITS PER C APITA1 

To give libraries some measures they can use for this purpose, PLA has developed 
OutputMeasuresforPublicLibraries. These are twelve ready-made measures that 
libraries can use if they fit the library's objectives. The following lists the output 
measures grouped under their related goal areas: 

GOALS MEASURED BY OUTPUT MEASURES: 

REACHING MAXIMUM # OF PEOPLE IN COMMUNITY 
LIBRARY VISITS PER CAPITA 
REGISTRATION AS A % OF POPULATION 

MAXIMIZING USE OF MATERIALS 

CIRCULATION PER CAPITA 

IN-L1BRARY MATERIALS USE PER CAPITA 

TURNOVER RATE 
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GOALS MEASURED BY OUTPUT MEASURES: (continued) 



PROVIDING READY ACCESS TO ^ 1ATERIALS 
TITLE FILL RATE 

SUBJECT AND AUTHOR FILL RATE 
BROWSERS' FILL RATE 
DOCUMENT DELIVERY 

PROVIDING INFORMATION IN RESPONSE TO QUERIES 
REFERENCE TRANSACTIONS PER CAPiTA 
REFERENCE CON IPLETION RATE 

PROVIDING INFORMATIONAL PROGRAMMING 
PROGRAM ATTENDANCE PER CAPITA 

If the library has an objective that corresponds to one of the output measures, it 
can use the following GENERIC OBJECTIVE FORM:To increase (name of output 
measure) fromicurrent number) to (desired number) by (date). For example: "To increase 
circulation per capita from 4.0 to 5.0 by October 1,1989." or "To increase registration as 
a percentage of population from 27% to 40% by October 1, 1989." 

If the library is seeking to attain some other end than is measured by an existing 
output measure, it will need to invent or locate a measure that will allow it to conclude 
whether it has attained the desired end or not. Often we can identify the needed 
measure by returning to the evaluations conducted in the Looking Around stage, where 
we assessed needs, to recall what data were used to document the need. The same kind 
of evidence may be used to assess the degree to which the need was met. 

Mary Kay Chelton has a useful discussion and illustration of goals and objectives 
on pages 55 in this Sourcebook. Adaptations of output measures to specifically measure 
children's services can be found in Douglas L. Zweizig, Joan A. Braune, and Gloria A. 
Waity, Output Measures for Children 's Services in Wisconsin Public Libraries, 
Madisom School of Library and Information Studies, University of 
Wisconsin— Madison, 1989. Selections from this report are found on pages 207-222. 
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This discussion of planning and evaluation has not addressed «*M*te. the step 
in the planning process that is called 'taking action." That was neglected in order to 
concentrate on the hard part-the part where the Horary Has to de.de what the 
needs are and what the library should be trying to attain. Takmg AcHon deals mth 
the means for responding to needs, and humans seem to find it easier to focus on mean, 
than to clarify our purposes or ends. 

The first item in this chapter presents an overview of an evaluation proem that 
can be used for planning and evaluation of a specific prefect or of the overall hbrary 
vrogram. In it, Robbins and Zaidz-s provide a guide for minimalist planning. Tfcis » 
flowed by an insightful discussion by Adele Fasich of the complexities, necemtus, 
and vossZties for evaluation of children's serves. Diana Young contu,ues these 
themes by presenting a structure for organising • , evaluation of services to cMren 
and by suggesting relevant questions to be asked about the various asspects of the 
ZL Zen Krueger illustrates the importance of being dear about what,tu you 
wnt to accomplish by providing examples of targets selected for serving ch,ldren. 
May Kay Chelton's Evaluation of Children's Services" instructs the reader m *sues 
and methods of evaluation and provides sample auestionnaires from actual 
evaluations that can be adapted to different settings. 

In order to support the evaluation of the contexl of children's services, the chapter 
on "Looking Around" is reprinted from Charles R. UcOure, Amy Owen, Doughs L. 
Zweizig, Mary M Lynch, Nancy Van House, Pl^mingandRoleSettingforPublu: 
LWraries^MamalofOptionsandProcedum, Chicago: American Library 
Association, 1987. This chapter is followed by a workfom designed to organize the 
iota gathered in the Looking Around process: "Information to Support Planmng for 
Children's Services." 

Overall, the intent of thk chapter of the Sourcebook is to provide a framework for 
understanding the planning and evaluation processes and to give some concrete 
suggestions for proceeding. 
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Introduction to Evaluation 



Jane Robbins and * louglas L. Zweizig 

Excerpted from Arc We There Yet?, 
School of Library and Information Studies, 
University of Wisconsin— Madison, 1988 

The term "evaluation" is likely to bring to mind the question, "How good is it?" But 
"Compared to what?" is the key question to ask. Evaluation implies a final judgment of 
value made by comparing the thins evaluated with some standard. Evaluation in the 
library requires that some characteristic of the library be observed and then compared 
against a standard. Which standard? There are a number of sources: 

. A national or state library association may set standards for 
libraries. 

. An outside evaluator may judge an individual library against the 
standard of other libraries the evaluator has knowledge of. 

. A library may compare itself with a neighboring or peer library 
("Keeping up with the Joneses"). 

. A library may evaluate actual performance in terms of deadlines or 
budgets used as standards. 
Standards that come from outside a library may not be relevant to it; thus a library 
must choose carefully. 

The purpose of evaluation is not just to know whether to feel good about some aspect 
of the library. Its purpose is to allow us to make better decisions about the library-to 
identify aspects that might be improved and functions that need to be speeded up or 
made less expensive. Rather than asking "How good is it?" this series of tutorials will 
ask "Are we there yet?"-* question the authors feel better reflects the uses to be made 
of the evaluation. This question sees evaluation as a process of checking on a regular 
basis to determine how much progress has been made towards a stated goal. 
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The question "Are we there yet?" grog's out of our working definition of planning, 
which is "a series of successive approximations towards a moving target." That is, 
planning involves a repeating process, a series, through which the library moves closer 
to its intended goal, the target. However, while the library is carrying out its plan, the 
demands being made on the library and the characteristics of its environment change; 
the target moves even as we attempt to approach it. 1 Assume, for example, that a 
library's target is to achieve an average processing cost of $325 per item processed. If 
the price of card stock increases greatly or if the library decides to go to an online 
catalog, the target of $3.25 will no longer be appropriate and will have to be changed. 
Pfenning's role is to select the target and to review whether that target has moved. 
Evaluation's place in the planning process is to periodically assess how much closer we 
are to the target. 

It is probably clear to the reader that our concept of evaluation is so intertwined 
with planning that both must be dealt with in evaluating any aspect of library 
operations. Knowing what to evaluate requires knowing what was planned. If you are 
not clear where you want to go, then how do you know when you have arrived? 

The Planning Process 

Although planning processes may differ in the number of steps required or the way 
the steps are labeled, most follow the same reasoning. For this series, we will use the 
steps found in ALA's publication, 4 Planning Process for Public LibrariestThe ways 
this process labels the steps in planning can readily be adapted for any type of library 
situation. The eight basic steps are: 

1. Assess community and library environment This step involves looking at the 
context of the library. What kind of community or institution do we serve? What 
kind of people are our users? What kinds of facilities are in the community? 

2. Evaluate current library services and resources. What is the library now doing? 
What is the library working with? What problems are now obvious? 

3. Determine the role of the library in its community. What kind of library should we 
be for this community at this time? 
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4. Set goads, objective., and priority. Different planning processes define the terms 
"goal" and "objective" differently. In this process, goals are general statements of 
the directions the library will move in; they are not time-bound and are intended to 
be finally achieved: "Our goal is to increase the use of the library." Objectives are 
more specific statements of what the library will try to achieve by the end of each 
planning period. They need to be clear, measurable in some way, and have a 
deadline: "Our objective is to increase users from 30% to 40% of our potential users 
by Sept. 30, 1986." Priorities are needed to assure that the most important objecbves 
receive emphasis during a particular planning period. 

5. Develop and evaluate strategies for change. There are many ways to increase the 
use of the library. This step in the process involves reviewing the means available 
,o the library and selecting the means to be used: "To increase use, we will produce a 
guide to the use of the library." 

6. implement strategies. This is the process of carrying out selected strategies 

7 Monitor and evaluate progress towards goals and objectives. As the plan is being 
' carried out, the selected strategies are monitored to be sure that things are 
happening as planned and on time. * the deadline for the objective occurs, an 
evaluation determines whether the targets were achieved; for example, whether 
users inaeasedfrom30%to40% of the potential user community. At the end of the 
planning pmod, a review of progress determines whether overall use is increasing. 

8. Review and update goals, objectives, and priorities. Throughout the planning 
process, ana particularly at the close of the planning cycle, goals, objectives and 
priorities are reviewed for appropriateness. Is increased use still an appropriate 
goal? Should the specific level of use to be achieved be increased or decreased? 
Should this area of service receive the same priority in the future? As progress is 
made and conditions change, these decisions need to be reexamined. 

Although the planning process presents a series of steps with dear beginning and 
end points, the process is in fact cyclical and can be begun at any point. A library could 
begin to set goals, objectives, and priorities for the next year (step 4), without a formal 
review of the community or the library. In the midst of this process, the library could go 
back to step 1 to learn more about the community, and so on. A planning process provides 
a checklist and a structure for making planning decisions, but a library may use the 
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process in the ways most suited to its situation. Although we most often think of 
evaluation occurring at step 7, it can occur at many points in the planning process. 

The remainder of this book will look at approaches to evaluation in a number of 
areas of interest to librarians: collections, reference services, library service programs, 
and personnel. In each of these areas, we will use the same process for evaluation. As 
we mentioned above, evaluation may occur at various steps in the planning process. It 
may be used to examine how well the library is achieving its ends (goals and 
objectives) or how well it is carrying out its means to those ends (the strategies). 

We have separated the evaluation process into seven steps for the sake of clarity, 
but this does not mean it is an elaborate process. The whole process could be recorded on 
a half sheet of paper (see Evaluation Summary, next page). 
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The Evaluation Process 

1. Determine the target area* 

The target area can refer to what you want to accomplish (effectiveness) or to how 
well you want to do it (efficiency). The process is very similar to determining a goal in 
planning* For example, a target area in reference may be the accuracy of answers to 
reference questions or the speed of service provided. In program evaluation, the target 
area may be what was learned in a workshop for library staff or how the workshop 
participants liked the workshop. A helpful question for determining the target area is, 
"What do you want to know about?" The answer may be that you want to know about 
several things. In such a case, each of those things is a separate target area. A 
description of each target area should be listed on a separate Evaluation Summary. 

2. Set the target 

For each target area selected/ a target needs to be set. That is, each target area 
identifies the aspect of library operations you want to evaluate, but the target is the 
specific standard you will compare your results against The target should be 
measurable (i.e„ expressed in a number), or at least observable (i.e., you should be able 
to tell unambiguously whether you've done it or not). Example of measurable targets 
are: redu <; cost to $5 per item processed by Dec. 31, 1986; achieve an average score of 85 
on the knowledge test at the end of the workshop; increase percentage of users from 30% 
to 40% of potential users by Sept. 30, 1986. Examples of observable targets are: complete 
budget proposals by June 30, 1986; obtain approval for policy manual by May 30, 1986. 
The target should be entered on the Evaluation Summary. 



Target 
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3, How will you know? 

Sometimes the information that tells you whether the target has been met is 
obvious. At other times, you must make decisions about the data required to determine 
how close you have come to the target How will you know, for example, whether costs 
have been reduced to $5 per item? If procedures are needed for the collecting of relevant 
data, this is the place to spell them out. For example: for the first week in December 
1986, the number of items processed will be recorded on a workform by the head of the 
Processing Department; the Accounting Department will provide personnel cost figures 
for the Processing Department; the number of processing materials (book pockets, date 
due slips, etc.) used per item processed will be determined by the department head; the 
associate director for technical services will enter these data into the formula and 
calculate the average cost of processing each item. As you design the evaluation, the 
procedures for gathering the needed data should be entered on the Evaluation 
Summary. 

4. Take a look. 

During this step, the evidence needed for the evaluation is gathered in order to 
produce the figure that corresponds to the target: the cost of processing items, the 
average score on the knowledge test, the percentage of new users recorded. The actual 
achievement should be entered on the Evaluation Summary. 



Target 


Actual 1 







5. How close are you? 

At this point, the comparison between the target and the actual figure is made. 
Judgment should be suspended, and only the facts recorded. The difference between the 
target and actual performance is usually calculated by subtracting the target from the 
actual figure, so that if the actual is greater than the target, the difference will be 
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positive, and if the actual is less than the target, the difference will be negative. 



Target 


Actual 


Difference 









6. So what? 

When the data on actual performance and the target have been recorded together, 
someone needs to make a decision about whether to act on any difference between them 
and what to do. If actual processing costs were $10 per item and the target was $5 per 
item, several kinds of decisions are possible: procedures in the Processing Department 
may be streamlined in order to reduce costs; processing may be contracted out to a 
processing center at a cost closer to the target; the target may need to be set at a 
different level if the original level did not anticipate a change in the costs of 
processing supplies. The decisions should be recorded on the Evaluation Summary. 

7. Rethink. 

Each step in the evaluation process may result in learning more about the aspect of 
the library being evaluated. The library may have selected a target area that is too 
difficult or too expensive to measure and may need to refine it. If the library does not 
have enough information to set a target, it may need to collect some data before setting 
one. Changes that occurred after the target was set may require rethinking the target. 
At any stage of the evaluation, a library manager may decide to go back and "fix up" 
some earlier step. This procedure is not only appropriate, it is necessary if the 
evaluation is to be useful. In recording the process and results of the evaluation, 
however, changes that resulted from rethinking should also be recorded so that a 
reader will know what you've done. 

Finally, although such a step is beyond the evaluation process itself, it is 
important to communicate the results of an evaluation. Staff needs to know the 
library's targets and how they are doing in approaching the targets. Evaluation results 
can highlight in specific figures for the tight-fisted f under the demands made on the 
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library and the success the library has in meeting such demands. The results can point 
out library progress in increasing use, the areas where personnel are overextended, or ^ 
the areas where the collection is unable to respond to demand. In short, the purpose of 
evaluation is to enable the library to operate better in the future by identifying areas 
needing improved performance or increased resources. 

Our arst lesson has presented a generalized evaluation model and related it to the 
planning process in libraries. Future lessons will apply the model to equation of 
collections, reference, library service prograns, and personnel. 



Notes 

im "The Science of Muddling Through," (Public Administration Review, 19 (1959), 
^fcTSlom put forth a remarkably similar definition of P^Jff 
Ccy-rr^Vis a process of successive approximation to some de^ objectives 
in wWchwhatte desired itself continues to change under reconsideration. 

2VemonE.Palmour, et al A Planning Process for ^\ k ^^ f^^ fo ^' 
also the more recent Charles R. McClure, et al Planning & ** *** for 
mTlZries; A Manual of Options and Procedures. Chicago: ALA, 1987. 
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Research and Measurement in Library Services to Children 



Adele M. Fasick 
Reprinted with permission from Canadian library Journal October 1978. 



Although service to children has been a part of public library activities for almost 
a century, there has been increasing concern in recent years about its future. At an 
American Library Association preference on children's work [in 19771, F. William 
Summers said that children's services, once the jewel in the crown of public library 
8ervice "may now be seen as a piece of costume jewelry that has grown a bit tawdry and 
shopworn over the years."* The same concern led the coordinators of Children's 
Services of Large Urban Libraries (Canadian) to prepare a series of recommendations 
designed to prevent the downgrading of children's services in public libraries. 

These recommendations, accepted by the Council of Administrators of Large Urban 
Public Libraries, express a hope that a continuing dialogue between coordinators and 
administrators has started .2 Much of this dialogue will surely touch on measurement 
and evaluation of children's services, because it is only by such evaluation that 
priorities for the allocation of library budgets and library personnel can be determined. 

Why have librarians engaged in serving children been put on the defensive about 
the way in which they evaluate their services, and what can be done to bring 
children's services back into the mainstream of library thinking? Much of the problem 
rests in explaining to the library community just what children's work is, why it is 
difficult to evaluate, and what part it plays in the lives of children, as well as in the 
total picture of library service to the community. 

To do this, children's librarians have to come to terms with the increasing demand 
for quantitative measurement of library services. Why are we being asked for a 
justification of services that we have been giving satisfactorily for many years? One 
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reason for this demand is surely the pressure of declining or stationary library budgets 
at a time of rising inflation, which sends costs spiraling upward every year. All 
publicly supported services— schools, hospitals, and museums as well as libraries— are 
being more closely scrutinized because of public demand that tax dollars be spent in 
ways the public sees as important. 

The existence of computers, which enable us to collect and store vast quantities of 
data, has made it possible to measure and evaluate library use in ways that were 
prohibitively time-consuming a generation ago. The pressure for budget justification 
and the existence of tools for manipulating statistics have led librarians in fields other 
than children's services to adopt a more quantitative approach to the measurement and 
evaluation of their services. Studies and measurements that began in scientific and 
technical information centers are being adopted by public and academic libraries. 

Services Hard to Evaluate 

Some librarians who work with children have avoided justifying their services on 
a quantitative basis by arguing that the work they do is not measurable, a claim that 
has led to misunderstanding and sometimes recrimination. There is some justification for 
this claim, but it seems clear that sooner or later children's librarians are going to have 
to adopt some forms of objective, quantitative measurements. The major question is 
whether these measurements will be the most appropriate ones, or whether they will 
be ones borrowed from other sources and applied willy-nilly to children's services. 

Although some of the reaction against quantitative measurement of children's 
services may be overly emotional, it is not true, as one of my colleagues once suggested, 
that "children's librarians are people who love children and hate statistics." There 
are some good reasons for protesting against the imposition of quantitative standards on 
children's work. There are problems in the evaluation of library services to children 
that simply do not exist in other types of library work. 

Education First Goal 

Some of these problems stem from historical conditions. When libraries first opened 
their doors to children at around the turn of the century, there was a reasonable 
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consensus about what the library was supposed to do for children. It. firs, task was to 
pn.vide educational books. Fiction was considered a frill to be discouraged. One pea 
Lr was that too much reading of library books might take children's minds oft their 
schoolwork The major requirements for children's books in librat.es were that they 
must be both educational and morally unimpeachable. 

The public library was viewed as a continuation of school more often than a 
supplenL. to it. Smce the vast majority of children left school by *e time dtey were 
fourteen years old, children's services were often designed to serve those young ; peop e 
wh0 had no further access to formal education. Children's libraries served children tn 
.heir last few years of school, after they had been taught to read. The ^librarj .was 
supposed tobegmaprocessofliMongeducationbydevelopingmchldrenahabttof 

reading good literature. 

Children's services at mat time were much easier to evaluate, ^«^< 9 
served a narrower age span of children in a comparatively homogeneous society. The 
Z£ goals wereltited, and few people questioned their value. The library stood 
done* I only social agency providing children witf , books, since most schoolwork 
was based on textbooks and encouraged little outside reading for classes. 

Goals Multiplied 

The goals advocated for children's library services today have grown from those 
comparatively simple beginnings. For example, the public library i, often expected to, 

. prepare preschool children for their school experiences by introducing mem to 

books and other media; 

. provide cultural experiences, such as drama, concerts, and film shows, for children, 
. help children adjust to the community in which they live, especially minority 
children and immigrant children, by providing materials and programs that wU 
teep them in touch with their homelands and introduce them to (their countries ) 



pre^'children for living in a multicultural society by providing materials and 
programs on various ethnic groups; 



UNIVERSITY OF WISCONSIN— MADISON 



Fttlck 
21 



9 

ERIC 



27 



• help children adjust socially and psychologically to the demands of growing 
up— and this includes children with physical, mental, and psychological 
handicaps that present severe problems; 

• compensate for deficiencies in homes and schools by providing services for children 
from homes without books, and for children whom the schools have not taught to 
read adequately; 

• provide programs and materials that will compete with television for the 
children's interest; 

• provide entertainment and a meeting place for children who have few other social 
outlets— especially during school holidays; 

• provide supplementary material for school projects, at times when the school 
libraries are unavailable; 

• provide a variety of materials and services that in one way or another will enrich 
children's lives (these may include toys, games, pets, craft classes, hobby 
clubs— the list is limited only by the librarians' imagination); and 

• develop in children a lifelong love of reading— this is an old goal that has not been 
discarded. 

Not only are all these goals supposed to be met simultaneously and in multimedia 
form, but services are also supposed to be brought to the children wherever they may 
be— in playgrounds, housing developments, or day-care centers. 

When we consider the size of the tasks suggested for children's services and the 
difficulty of evaluating any one of them, it is no wonder that librarians protest that 
they cannot measure their services and present them in neat statistical tables for the 
annual report. 

Children as Clients 

As if the multiplicity of goals were not enough, services to children also present 
problems of measurement because children as clients present some particular 
difficulties. First of all, many of the children who use our libraries cannot be expected 
to tell us what they want or let us know how satisfied they are with what they get. 
When they first come to the library, they may be just beginning to talk, and completely 
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unable to articulate thelv reactions. As they grow older, they can tell us what they 
think of the tnaterials and programs we o«er, but as a rule they do not know what el* 
might be provided. 

Generally children don't read book reviews, and few of them visit bookstores; as a 
result they have little idee of what is available. For the most part they accept or 
reject the library as it is, without believing they could influence what it might be. We 
found this war true even of ten- to twelve-year-olds in the survey done at the Regina 
Public Library. 

As an added complication, children do not necessarily know what is good for them 
They are subject to the direction of other adults, chiefly parents and teachers, who set 
limits on the kind of materials and se.vices they should get. This means there is 
another group of people who may not use the collection directly, but those ideas must be 
taken into account when evaluating services. 

Besides this, public library services in most communities are supplemented by 
school library services. The ideal relationship between these two institutions may 
vary from one community to another, but there certainly has to be some coordination of 
services. There is no comparable institution to take into account when evaluating public 
library service to adults. 

Research . 

This discussion of the difficulties of measuring children's services may paint a 
bleak picture. Children's services can be measured and evaluated in an objective way, 
but in order to measure services, we need many more facts than are available at present 
This is where we come to the issue of research and what it can do m the field of 
children's work. 

Can research serve a useful purpose? That question is still often asked. The answer 
is "yes," but only in specific and limited ways. Research, in the broad meaning or a 
careful, systematic search for new facts, should help in the task of measuring and 
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evaluating children's services, but it is not a panacea that will solve all library 
problems. 



Research studies can answer questions about what is being done now in public 
libraries. Despite years of experience, we still cannot answer questions such as: 

How much of the community are library services reaching? 

How much time is being spent on materials selection, on helping individual children, 

groups of children; how much staff time is spent in preparing and presenting 

programs? 

How well are refr* jnce questions answered? How much time does it take to do reference 
work with children as compared to adults? How much reference work deals with 
school-related projects and how much is generated by the children themselves? 

Which children do not use the library? Might they become library users under different 
circumstances? Are the changes necessary to attract them compatible with serving 
children who already use the library? 

The answers to these questions are basic in evaluating ho .v well the children of a 
community are being served, yet most libraries cannot answer them except by guessing. 
These questions are limited enough to be answered for a particular community on the 
basis of a survey. Such a survey would be time-consuming and therefore expensive, and 
could not be undertaken by every library. But even a limited number of surveys in 
communities of various sizes and geographic locations would yield valuable 
information that might be applied more widely. The Regina children's library study 
was an attempt to answer questions of this type. 

Complex Questions 

There are other pressing questions that would have to be studied over a period of 
time, or over a large number of communities, in order to give us useful answers. Some of 
these questions are: 
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Do children who use the library when they are young tend to grow up to be library 
users? 

Do children who attend library programs become long-term library users? 

Do children move from reading low-quality books (e.g., Enid Blyton or the Hardy Boys) 

to reading high-quality children's books? 
Are there differences in services given in libraries that have children's coordinators as 

compared with libraries that do not? Are there differences between libraries with 

children's specialists and those with generalists? What are these differences? 
Do outreach programs in playgrounds and housing projects eventually bring children 

into the library as they grow older and become able to travel? 
Do communities with highly developed school library systems have more or fewer 

public library users? Are the two types of libraries used differently? How? 
How is adult use of public libraries affected by children's services? 

These questions are not easy to answer, and research studies designed to answer 
them will be lengthy and expensive. Nonetheless, we are now making policy decisions 
based on assumptions *e have made about the answers to some of these questions, when 
in fact we cannot demonstrate that our assumptions are correct. If we are ever to build up 
a body of knowledge on which to base our planning, a start must be made on finding 
answers. 

Limitations of Research 

Some of the disillusionment with research studies comes from the fact that 
librarians are hoping to have questions of value, or policy decisions, answered by 
research. Questions such as: "Should we buy series books, or comic books, or whatever? 
Should we integrate our nonaction children's collection with adult nonfiction? Should 
we have film shows on Saturday afternoon?" cannot be answered by a survey unless the 
objective of the change is spelled out 

The question of whether or not to have film shows on Saturday afternoon might 
have different answers depending on whether the unstated goal is "If we v.ant to get 
more children into the library to see what we have to offer, should we have film shows 
on Saturday afternoon?" or "If we want to encourage children to read and use our 
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collection more, should we have film shows on Saturday afternoon?" or perhaps, "If we 
want to get more older children into the library should we have film shows on Saturday 
afternoon or on Friday evening?" The librarian has to think through the purpose for 
offering any particular program or service before a reasonable research question can be 
framed. 

There are also some questions for which research has no answers. An example wouid 
be "Is it better for a child to watch The Red Balloon or to read The Hobbit?" That 
question might be answered differently by different people. There probably is no final 
answer, and certainly no research study will decide the issue. 

Another kind of question that research will not answer is that dealine with 
decisions between individuals, such as which of two candidates to promote to the job of 
children's coordinator. The issue of which is more important for a children's 
coordinator, an M.L.S. degree or years of experience, may eventually be answered in 
general terms such as "coordinators with five or more years of experience tend to do a 
better job than those without experience," but it can never answer which of two 
particular individuals will do a better job in a specific situation. Research, in other 
words, doesn't eliminate the need for making difficult administrative decisions. 

These, then, are the most important limitations of research. It cannot answer 
questions of value— cannot tell you what your goals should be, but only indicate how 
you might achieve particular goals. It cannot tell you what will happen in a specific 
case; it can only indicate overall trends or probabilities. A research study might say 
that circulation would probably be increased if comic books were added to the 
collection, because 80 percent of children surveyed said they liked comic books. It cannot 
tell you whether or not you should try to increase your circulation in this way, and it 
cannot tell you whether eleven-year-old Carol Smith will use the library if comic 
books are there. 

Developing Research 

As long as we have reasonable expectations of what research can do for children's 
services, we have to acknowledge that building up a body of such research would be of 
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value for measuring and evaluating services. The next question that arises is how this 
body of research can be developed. There are three major groups of people who have 
responsibilities toward developing research-the staff working in libraries now, the 
professional associations, and the library schools. 

How reasonable is it to expect library staff members to conduct research? There is 
only a limited amount that can be done by individual libraries, because of the severe 
shortage of staff time. In most children's departments, the staff are fully occupied for 
all their working time. It is unreasonable to suggest that they take on extended 
research work unless additional staff is provided, or unless other tasks are taken away. 
It might be possible, for example, to stop presenting story hour programs for a month or 
two in order to have some time free for a special project. Another possibility is to hire 
students or part-time help to collect data for a period of time under the supervision of a 
staff member. 

There are some limited studies, especially those that duplicate studies already 
done in another community, that might be run by a large library system using its own 
staff. It might also be possible for library personnel to collect data in a standardized 
way as part of a larger research project, directed from outside their individual library 
system. These will necessarily be limited projects-useful but not sufficient to provide a 
large body of research. 

The major role of library staff is likely to continue to be to read research studies, to 
use them as the basis of discussion in staff conferences and meetings, and to try to see 
how these studies might apply to their own library situations. If they also write up 
their reactions to the studies and the results of their attempts to apply them, they wUl 
be contributing greatly to a body of knowledge about librarianship. 

Urge libraries and regional systems can contract out research rather than use their 
own staff. Such research must be published and disseminated in order to make it useful 
to the profession as a whole. This should be seen as an obligation by those library units 
large enough to support such studies. 




Professional associations have an important role to play in developing and 
maintaining a usable body of research. The publications of professional associations are 
the major channel for disseminating research findings. The development of refereed 
journals for research would be a valuable step toward upgrading the quality of 
professional publications. 

Another role of associations is to use conferences as a forum for the presentation and 
discussion of research findings. Perhaps for the next few years in particular, conference 
discussions among children's librarians ought to shift away from emphasis on materials 
and programming. Despite the importance of these services, 't is probably more 
important at the moment to look at the overall questions of what we know about 
children's services and what we need to learn. 

Still another role professional organizations can play is to serve as coordinating 
agencies for fund raising. Since most public libraries cannot fund major research studies 
on their own, regional and national associations may find it possible to bring together 
libraries that can contribute to cooperative research, 

Library Schools Have Major Role 

Although librarians working in the field and library associations have roles to 
play in developing research, neither of these groups can take major responsibility for 
conducting research. It has often been said that librarians ought to do more research, but 
when we look at other professions, we can see that this expectation is probably 
unrealistic. If medical research were considered to be the responsibility of practicing 
physicians, little would likely be done. In most professions, the professional schools 
and university faculty in allied fields conduct the basic research. This is likely to be 
the most fruitful pattern for librarians to follow. 3 

The library schools have a pool of faculty members trained to do research. They 
also have affiliations with universities, which contain faculty members in other 
disciplines who can offer specialized help in aspects of research. The responsibilities 
of the library schools are twofold. The first is to teach students enter:*g the profession 
to know about and to appreciate the value of research in carrying out their library 
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responsibilities. Second, faculty members should carry out research that will enrich the 
materials they are teaching and will also help practitioners m the field. 

There are a number of factors that have limited the amount of research that is 
done Some of these are internal to the schools-the burden of time-consuming 
committee work, supervision of students doing practicums, lack of clerical assistance to 
help with day-to-day routines of teaching and keeping up with the library field. 
Others are external. Lack of funding is certainly important, but probably the most 
important single reason for a lack of research on the part of library school faculty 
members is the feeling that research is not valued by the field and will not be utilized. 

The profession often seems more eager to have faculty members attend conferences, 
run for association offices, and organize yet one more workshop than to engage in 
research. These activities are important, but they can be dangerous in that they offer 
an instant appreciation and feedback that may entice people away from the slow, 
lonely process of research. In the long run the library profession, like any other, will get 
more of what it seems to value most. Librarians must be sure that rewards are allocated 
for activities that are most important for the profession. 

Despite these difficulties, the surest way to build up a research foundation for 
Ubrarianship, which will enable us to measure and evaluate services in a useful way, is 
to make use of the facilities available in library schools. With the growth of doctoral 
studies programs, there is in addition to faculiy members a group of students engaged m 
learning about and carrying out research. While doctoral dissertations cannot be 
developed merely to supply the answer to a question puzzling some library, these 
dissertations should offer to the field research findings that will gradually build up a 
core of usable knowledge. Professional associations might offer more encouragement to 
doctoral students by occasionally inviting them to talk about their research and discuss 
its implications for libraries. This kind of contact would be useful for both students and 
practitioners. 

Library schools need to make research available in easy-to-find form. Doctoral 
research should be published not only as a complete dissertation-often available only 



UNIVERSITY OF WISCONSIN— MADISON 



9 

ERIC 



3^ 



on microfilm— but also written up as one more journal articles, which are more widely 
distributed and easier to read. 

If faculty members at library schools are to fulfill the two obligations of using 
research in their teaching and of working to develop a body of research for the future, 
they must coordinate their efforts with practitioners in the field. The report of the 
children's coordinators mentioned earlier spoke of setting up a continuing dialogue 
between coordinators and chief librarians. If this dialogue is to be effective, it should 
include library schools. Only by bringing together all sectors of the profession, each 
playing an appropriate role, will we be able to overcome the difficulties of measuring 
and evaluating children's services and be able to develop reasonable goals and 
objectives for service to children. 
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Evaluating Children's Services 



Diana Young 

Reprinted from Public Libraries, Spring 1984, p. 20-22. 

When was the last time you stepped outside the children's department and really 
looked as an outsider, at the children's room, the materials collection, and the services 
offered to children by the library? Is the library meeting current needs? Expectations? 
Are there more or fewer children using the library than two years ago? Why? Is usage 
of library materials increasing or decreasing? Why? 

Evaluation means looking at children's library services and determining how best 
to meet library goals and objectives. It means reviewing your library's community 
analysis and reexamining your short- (and long-) range plans. It means determining 
strengths as well as areas that require change. The following suggests points for 
consideration in the evaluation of children's services. 

Facilities 

The first thing a patron sees is the facility. If the client enterr *sy, messy area or 
an overly neat, silent one an attitude about the library and its sta . .ormed. In order 
to determine the condition of your library's children's area, you will need to step out of 
your role as librarian and assume the role of patron. Consider these points: 

. What is your initial impression of the area? Is the area "receptive," or must you 
take into account factors only a librarian would understand? Is the condition and 
arrangement of the area physically inviting? EasUy supervised? Does the area 
exhibit a caring cmld awareness and concern? 

. Is the area inviting to children? Are shelving, bulletin boards, display cases, 
stools, and other furnishings suitable for a child physically? On a child's eye 
level? Are fresh, suitable colors used thioughout the area? Are there bright posters, 
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mobiles, or other child-enticing displays to draw the child into the room? Does the 
room suggest the library purpose—playground, study hall, informality? 

• Is the area carpeted? Or is there a rug in the picture-book area to cut down on noise 
and encourage children to sit on the floor? To enjoy the library? 

• Is the a*ea inviting to parents? Will parents want to bring their children into the 
area for a first visit? Are there welcoming displays of materials? Are there stools 
for pregnant mothers to sit on while helping preschoolers? Are children allowed to 
be children? Is the librarian in control of the area or are the children? Are 
unattended older children using the area designated for young children? Are 
unattended young children allowed to wander and disturb older children? 

• Is the room safe? Are unused outlets covered? Are unsupervised audiovisual 
materials well out of the reach of preschoolers? Will the furniture tip over if a 
young child stands on it? Can paperback or magazine racks, display cases, or pet 
cages be easily pulled over? Are th aisles crowded? Are small objects kept out of 
the reach of young children? Are unattended children left unsupervised in the 
room? Are adults wandering throughout the area without reason or purpose? 

• Is the room orderly? Are display materials, bulletin-board materials, and other 
little-used things stored out of sight? Are the materials neatly arranged (not in a 
strict, uninviting, everything-in-its-place order, but neatly, considering child-use 
order)? Are materials shelved daily, or must patrons plow through stacks of 
unshelved materials? Are puzzles and other toys picked up daily? 

• Is the librarian positioned in an accessible place? Does placement of the service 
desk encourage questions and suggest a willingness on the part of the librarian to 
assist? 

• Is the area suitable for use hy handicapped children and/or parents? Is there space 
for a wheelchair between ti e shelves? In the reading area? 

There are many other points to consider. Take time to make a list of the area's 
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strengths and weaknesses. Make a plan of action and follow it. Many times the task 
will seem overwhelming, but if a reasonable plan of action with accomplishable goals 
is set and followed, the task is not so great. Sometimes a minor change will make a big 
difference in the welcome exhibited by the room and staff. 

Materials Collection 

When and how a patron determines library needs can be directly attributable to the 
condition of the collection. It matters not whether the materials are available if the 
condition of the collection suggests that the materials are undesirable. Weeding is an 
important part of maximizing circulation, and its neglect can quickly lead to circulation 
l,ss. Some points to consider in examining the condition of the collection are as follows: 

. What is the physical condition of the collection? Are there worn-out materials on 
the shelf? Are any materials dirty? Do they have torn pages, multiple crayon 
marks, broken bindings, missing pieces, scratches, or other broken parts? Would a 
child be frustrated by checking out a book, puzzle, record, tape, or film that doesn't 
work properly? 

. What is the currency of the collection? Are the reference materials up-to-date? 
The nonfiction materials relevant? Are current topics covered in fiction materials? 
Are new materials on a wide range of top* cs added regularly? Are old favorites 
regularly replaced? 

. Is the collection filled with unneeded "gift" materials? Are gift materials treated 
with the same care and concern as purchased materials? Are they retained because 
they are "gifts"? 

. Are materials available to children? Are children's materials located in an area 
that gives children priority? Are materials within easy reach of their intended 
audience? Are materials for young children on three-foot high shelves or in low 
bins? Are materials for older children on sixty-inch (or sixty-six-inch) shelves? 
Are records in low bins? Are paperback and magazine racks at a child's eye level? 




0 

ERIC 



3' 



Services 

Promised services need to be delivered. It does not help the library's image to offer 
and promote services that cannot be provided. Sometimes the library must reexamine 
services to determine which ones meet library goals and objectives and which ones 
unnecessarily consume staff time. This is perhaps the most difficult point of 
self-examination. It touches sensitive areas and programs that often provoke 
disagreement between children's librarians and the library director. Planning and 
awareness ensure that points of compromise enable the staff to continue to meet the 
library needs of children. To determine the best service possible, it will be necessary to 
examine the services aiready offered to children by other agencies in the community 
and to determine the best method to meet children's library needs. Services for the 
purpose of this article is defined as programming — for the library as a whole, for the 
main library, for branch libraries— and as outreach. 

Programming 

LIBRARY AS A WHOLE 

• What services are currently offered to children? Are they successful? Are they 
meeting children's needs? Are there other agencies offering these services? Are 
they better funded? Better staffed? Better able to meet the needs of the children 
being served? 

• Are all the community's children being served? How many children are there in the 
community? Which children don't have access to existing services? Whose role is it 
to provide service to children not being served? How can the neglect of these 
children be remedied? 

• What is the library's mission? What services could the library best offer children? 
What is the goal of public library service to children? What objectives are in place 
to meet the goal? Should the goal be reevaluated? Should the objectives be 
reevaluated? Should the goal be changed? Should the objectives be changed to 
better support the goal? 
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. What services can only be offered by the public library? How are those services 
being addressed? Does the pubUc library need to address them? 



. What patron input has been provided for in planning? How can patrons participate 
in the library's planning for children? 

MAIN LIBRARY 

. What services can best be offered by the main library? Who does the main library 
serve? How many children have easy access to the main library? Who are these 
children? What are their needs? Does the main library serve mostly parents who 
pick up school assignments and their children's leisure materials? Which 
objectives that support the overall goal for library services are best met at the main 
library? What is the main library's goal for library service to children? What are 
its objectives? 

BRANCHES 

. What services are offered by the branches? Should each branch be indi idually 
considered? What is the client makeup for each branch? Does it vary from branch 
to branch? What is the socioeconomic status of the client area? How many children 
are in the client area? What are the children's library-service needs? Can children 
visit the library without the aid of their parents? Can the objectives for one branch 
be different from another? How do the branch objectives fit into the overall library 
goals? What is the branch goal for library service to children? 

OUTREACH 

. What children's library services are currently being offered through outreach 
programs? Have the goals and objectives for outreach service to children been 
reevaluated when new branches were added to the system? Which outreach goals 
are still viable? Which are now best met by the new branch? How does the current 
service meet the overall goal? Should the goals and objectives be reevaluated? Is 
the outreach service a tangible part of the system's total plan for children's 
service? 
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The staff of every library system should take stock from time to time of who they 
are, where they are headed, and the best way to get there. With the fast pace today 
and the press of vocal demands for service, mar.v librarians find little time for self 
examination. Yet, this is perhaps the single, best way to ensure that there will alwa ;s 
be library services for children. If children's librarians don't stop long enough to 
evaluate children's services, who will? The library director? The county commissioner? 
The library board? Or a disgruntled patron who will bring about swift and often 
ill-conceived changes? How much better library services for children will be if they 
are planned for and regularly evaluated. Stop today and look as an outsider at your 
library's children's area. 

For Further Exploration 
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What is Good Children's Library Service-How Can You 



Adapted from a speech by Karen J. Krueger 



We had gorgeous weather in Madison this weekend and for the first time this year I 
was able to get out in the yard, dig around in the dirt, and get some plants planted. 
What I was waiting for all winter finally happened. Gardening is important to me and, 
as I finished the weekend, I decided I would begin my presentation today by talking 
about gardening. To throw everyone off balance just a little bit more, I decided to also 
begin by telling a story, which is really dangerous in a group of people who specialize 
in service to children and young people. Nonetheless, I am going to tell a story about 
gardening. 

This particular story is about a young woman who had lived in apartments for 
years and years and one day decided she was going to buy a house. After careful 
searching she bought a nice house in a suburban neighborhood. The house was situated 
on a wonderful lot, large but with not too many trees or interesting plantings. She 
wanted to have a beautiful yard so she hired a landscape architect. The landscape 
architect was recommended by friends and neighbors. The result was a complete 
landscape design. The plan was impressive. It was all drawn to scale and looked very 
professional, like something out of House Beautiful. 

She took that plan and spent years trying to put it into place. Every year she did a 
little bit more so that, while she could not do it all at once, over a period of years she 
accomplished a great deal. The results were spectacular; she had a beautiful yard. It 
was the talk of the neighborhood. People exclaimed over what she had created. Like a 
house decorated with thousands of Christmas lights, her house attracted Sunday 
drivers, gawking at the profusion of color and variety of flowers. She found that she 
was very proud of this accomplishment. The many compliments she received made her 
feel good. 
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In addition she won an Orchid Award from the Chamber of Commerce for her 
beautifully landscaped yard. There was a photo in the newspaper. As part of the story, 
the reporter asked her to define good landscaping. She responded by repeating what 
she had read about landscaping and by repeating what other people had said to her 
about her yard. When the story appeared in the paper, she read and re-read 
it— somehow it didn't seem right. She puzzled off and on about this for weeks. Much to 
her chagrin, she eventually decided it was a failure; it wasn't good landscaping at all. 
The reason—it wasn't what she wanted! She had not articulated to the landscape 
architect what she had in mind (hadn't actually articulated it to herself, for that 
matter). The landscape architect, like some library consultants, had his own agenda 
and proposed what he thought was important, what he thought should be included in 
the plan. It wasn't that the person she hired wasn't talented— he was wonderful! But 
he came in and designed a plan that had all the right elements but was for the wrong 
client. The plan did not accomplish what was important to her. 

What was her yard like? It had lots of roses and hundreds of bulbs (many were 
semi-hardy). It was a showplace for dahlias, and included a number of fruit trees, 
edible berries, an herb garden, and a cutting garden with exotic varieties of flowers. A 
well-landscaped yard, by contrast, would not have included roses, semi-hardy bulbs, or 
fruit trees. It would have included a variety of plants that once were in the ground did 
not have to be dealt with for many years, did not need to be pruned, sprayed, or 
pampered. 

In other words she wanted landscaping that required low maintenance. She also 
would have preferred shrubs and other plants which attracted birds. There were many 
things that she wanted that she did not get, and things she got she did not want. How 
would she have known this? How would she have articulated this and decided what it 
was she wanted before the plan was put on paper and she began implementing it? 

She would have done a needs assessment— assessing her own needs. What would 
she have observed? 
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She is a teacher; she makes a modest amount of money. She has her summers off and 
loves to travel. She likes birds. She is very active in the community (little theater, 
AAUW, Friends of the Library, her church, etc.) She has lots of hobbies, very little 
time. She is single, not fond of cooking. Convenience is important to her. 

In comparing what she wanted versus what she got, we observe that although she 
did not want to spend a lot of time in the yard, in actuality she spent weekends and 
many evenings pruning, weeding, spraying, digging up and replanting bulbs, etc. 
Although she wanted to spend a modest amount of money, when all the costs were 
added up for her grand plan she actually spent over $5,000. As a result, for three 
summers in a row, she did no traveling because she was paying for her landscaping. 
Although she preferred to spend as little time in the kitchen as possible, her Protestant 
work ethic compelled her to put the bounty from her garden to good use. As a result she 
spent a great deal of time canning and freezing, and trying to use the many herbs she 
now had available. 

Because she wasn't clear about what she wanted to accomplish-what her ends 
were -9he got something entirely different. Evaluation was useless because she was not 
clear about her objectives. When she tried to evaluate, she did so on the wrong 
criteria-what other people thought, the award she received, textbook descriptions of 
the ideal landscape, etc. 

With the conclusion of this story, I believe I have told you most of what I know 
about evaluation. It sums up how people make mistakes in evaluation because they 
focus so much on the information they get back. Librarians get feedback from many 
different sources: we conduct surveys, collect data, hear people's comments, have all 
kinds of information coming back to us about what we do. What we do not do is 
articulate what it is we want to accomplish so that the feedback and the information 
that keeps pouring in makes some sense and we have a context within which to place it. 

The title of this talk is "What is Good Children's Services-How Can You Tell?" 
There are some definite links from my landscaping story to children's services. 
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Assuming I know/ as a library director, what my library wants to accomplish, how 
do I get information which will tell me if we have done what we said we wanted to? 
There are a number of ways. Observation is certainly the easiest. I can walk through 
the library and I can listen to people when I am outside the library. * can observe many 
things. In the library, smiling faces, numbers of people, attractive *ess of the children's 
department, are all important. Outside the library I hear comments— both solicited 
and not — from people willing to tell me what they think of the library. Because we 
have a computer catalog gone bad, people usually tell me they don't think a lot of the 
library these days. These comments are very helpful: the number of suggestions we get 
in the suggestion box, any letters to the editor of the local newspaper (we had four 
regarding our policy on not admitting children to the puppet plays after the show 
started.) I can look at the noise level in the children's department; the higher the noise 
level, the better. Well, perhaps. Other criteria for evaluation might be how quickly 
the carpet wears out in the children's department compared to the other departments, 
how often there are empty chairs, or whether there are ever time periods when the 
microcomputers are not being used. Observation is very easy, it is cheap (usually), and it 
does give you information to help you make some evaluation decisions. 

Direct feedback from users is certainly important to look at. We did a user survey 
last year and we are in the process of doing a citizen survey. I have worked with 
libraries which have done student surveys, and they can also be very helpful. Getting 
this direct feedback is somewhat time-consuming, can be costly, and takes a lot of 
planning and careful construction of the survey instruments. But it is very rewarding. 

Comparison with other libraries is another evaluation method, and this is one I 
think has to be done very carefully. The Public Library Association has begun to assist 
in this area through the Public Library Data Service (PLDS). The PLDS is part of the 
Public Library Development Program— in addition to the planning and role setting 
process and output measures. PLDS is a beginning, albeit very modest. There is nothing 
in the PLDS about children's services, for instance. But current comparative data is a 
step in the right direction. Wisconsin has something called the Wisconsin Library 
Service Record and I am sure other states have similar publications. These statistical 
type reports allow you to look at what other libraries are doing. I do urge caution, 
however, because we should not be be competing with other libraries; comparative data 
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can put your services and performance in context, and perhaps alert you to your library's 
strengths and weaknesses. 

The most common source of information to help with evaluation is data, both input 
and output measures, on our own library. Ideally these data have been collected in the 
same way over a period of time. Data such as the number of volumes we own, the 
amount of money we spend on children's materials, number of staff , etc. Within our 
children's materials budget, how much are we spending on picture books, board books, 
toys, microcomputer software, and other types of materials. How has that changed 
over the years? How has staffing changed, especially in comparison to circulation, 
number of programs, etc.? Output measures such as circulation per juvenile capita, 
turnover of different areas of the children's collection (such as picture books, easy 
readers, and board books),the number of programs per juvenile capita, the amount of use 
our children's microcomputers receive, the amount of penetration we have made into 
the community. Usually we have lots of statistics available, a lot of data that can be 
used for evaluation purposes. The challenge is to have the right data and to use it 
wisely. 

What I find frustrating is the difficulty of distinguishing statistics and data as 
methods of evaluation distinct from the evaluation itself. Data assist in the 
evaluation, but by themselves, those observations, those surveys, those numbers tell you 
nothing. If I know that our circulation per juvenile capita is 85 and I look at other 
libraries and I see that the median for most other libraries our size is 62, 1 could 
conclude that we are doing great. That is the wrong approach. The number itself 
describes our performance. It doesn't tell me anything unless I know what number it is I 
want to reach. What is the target? 

Like the teacher with the elaborate landscaping, we need to know what we want. 
What do we want our circulation per juvenile capita to be? Do we want it to just keep 
going up? Do we want it to always be better than the neighboring library? Not 
necessarily. I think that the key-^nd I will say this over and over again-is being 
very clear about what it is you want to accomplish. 
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I guess the next obvious issue is how to set the targets, how to know what those 
targets ought to be? I will use targets and objectives interchangeably. Deciding on an 
appropriate target does not sound like it should be too difficult, but I find that it is. It's 
challenging and complicated. 

In the past, what we have done to help us set those targets is to rely on standards. 
Somebody else, like the landscape architect, set the targets and decided what libraries 
were going to accomplish. In preparation for this talk I dug out the old 1966 national 
standards and thought it would be fun to look back to see what some of those targets 
were that were set for us. Here's one; at least 5% of a library's annual additions should 
be materials of specific interest to young adults. Another: the proportion of juvenile 
holdings in the total system should be a minimum of 25% to a maximum of 40%; no more 
than 1/2 of these should be new titles. In general 2/3 of the annual additions for 
children should be replacements or duplicate copies of older works. Even the largest 
systems should not exceed 1,000 - 1,200 new titles annually. 

Working with these targets, set for us, was very easy. We collected the data and 
asked ourselves how many new titles we added to the collection the previous year. If 
we met the standard we could feel good that we were doing a good job. It was very clear 
cut. That is one of the reasons librarians like standards. There is much less ambiguity 
than exists in planning. But much less relevance as well. What is important is planning 
and evaluating in your setting for your library with your clientele. 

How do we pick targets? What I am going to tell you today is what I know about 
this topic. I hope that when I finish you will add to that knowledge base and tell me 
what you know, so that I can go back and do a better job when I have to do it the next 
time. Selection of targets generally occurs midway through the planning process— you 
have done your looking around and your needs assessment; you have a mission statement 
that doesn't say you are going to be the "Center for the Life of the Mind"; you have 
looked at roles; you have articulated what your priorities are, and you have goals. You 
are trying to articulate a specific target, a specific objective that is measurable, that 
says exactly what it is you want to accomplish. You know what the area is, but you are 
not sure what the specific target should be. 
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To be perfectly honest, the way we have done it most often is to pick a aumber (if it 
is a quantitative measure) that feels right. 1 was glad for the definition of planning as 
"successive approximations to a moving target" (Robbins and Zweizig), because that is 
exactly how we feel all the time. We aren't sure, we don't know, we h*ve a hint, we 
have talked with people, we have looked around, we look at what else we have to do, 
and we say, well, how does 60% sound? 

To help arrive at a number we look at our state data (ours does not include children's 
services information, however) and at neighboring libraries. We call them up or talk to 
them at workshops and conferences and ask lots of questions. We have to be realistic 
about the community we serve and to set a target that is realistic given our community. 
It may be different than the community next door so we do not try to pick the same 
target I review the reality of our own situation it. terms of budget, how much money we 
have to wor: vith, the staff we have on board, their skills and capabilities. All of 
those thing* >.;er into the realism factor when we pick a target. Also I look at the 
combination of roles we have chosen. We are a Formal Education Support Center, a 
Popular Materials Library, a Reference Library, and a Preschoolers' Door to Uarning. 
As a result we find it more difficult to select targets as high as we would like in some 
areas. Doing three or more roles well will always be more difficult than doing one role 
well. I also look at past performance, at trends-how have things changed over a five 
year period? 

All of these data and bits of information enhance and facilitate the selection of a 
target-a target that feels right to you and the staff. Perhaps in the discussion period 
some of you can share how you select targets, but that is in essence how we have done it 
at the Janesville library. 

I thought next I would give you some examples of decisions we made at my library 
regarding targets. We did choose as two of our roles, Preschoolers' Door to Learning and 
Formal Education Support Center. Both are secondary roles for us. We have a goal that 
says the library promotes and encourages reading and reading readiness skills among 
preschool children in the Janesville service area. We wanted one of our objectives to 
include market penetration, in other words, the number of preschoolers we reach. We do 
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not know at present the extent of our success; or put another way, the extent of our 
failure. We were concerned that we were teaching too few. We have a very active 
children's program, but we know that our sta<yhours, even when offered three times, 
reach relatively few children. Other programming, such as puppet f dws, and Mother 
Goose programs, are only able to reach a limited audience, We estimate we have about 
5,000 preschoolers in our service area. The question for u* was, what is the target? 
What percentage of the preschool population do we warn to be able to reach? 

We had no comparisons with other libraries. We had no current data and no past 
data. We took into account the fact that Janesville is a blue collar community, home to a 
large General Motors plant. In most families, both mother and father work outside the 
home and do not have much spare tin*. Many may not be readers. We thought it would 
be harder for us to reach a high target in this particular objective than for a community 
Nkt Madison which it highly educated and very aware of the importance of reading. 
Nonetheless, since \ve did not know current performance we tskad ourselves what we 
would like to afptire to. We chose a target of 75%. Of course, < me of the very first things 
we need to do i* find out what percentage of preschool population we are currently 
reaching I Author's note: Since this talk was given, the results of our citizen aurvey 
hava beet* compiled. According to our definition ot reach, which if that a preschooler 
has visHao a public library in the past year, we "reach" 71% of the preschool 
population in the City of Janet ville.] 

Another goal at my library i* to offer yunti% pmcyAe agr* 12 to 17 a variety of 
programs and services de#*gned for their developing needs and interests. As part that 
goal we have art objective to produce a 75% author and title fill rate for young adult* 
How did we choose that? In this case we looked at our adult figures because we do have 
figures for the adult population. We decided we could not achieve the same 
performance as we achieve with adults because we have provided very few services to 
teenagers over the years. Our collection is small and we devote only five hours a week 
of staff time to young adult services (now being increased annually by five hours a 
week). As a result of this comparison with our adult author and title fill rate, A^e scaled 
down our young adult target because of what we perceived to be reality. Over a period 
of time we will keep setting that target higher as we see what we are able to achieve. 
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The last example target I will share with you is that we hoped to reach 60% of the 
12-17 age population with library service for personal as weU as school related 
purposes. We know that we get lots of teenagers in the library; our user survey showed 
that\nd we can observe it. But we want to be sure we are reaching these teens for then- 
personal needs as weU as school related needs. We do not want them to just come in, do 
Lr homework, and leave. In terms of setting .he Urge! f< this, we I 
we we* emphasizing the formal education support center role we would be initiating a 
lot of contacts with the schools, and would have ample opportunities to entice middle 
arci high school students with other services not related to coursework. We believed we 
hadagoodchance of performing higher in meeting young adulfs personal needs thana 
library might if formal education support center was not one of their chosen . roles. 
However, we also knew that with only five hours a week of a young adult kbranar. s 
time, this was also going to be a challenge. [Author's note; Our recent ^survey 
showed that in the past year, 74% of the 12 to 17 year-olds in Janesvule used the 
library for personal as well as school related use.) 

A question we keep asking ourselves in setting targets is whether the target should 
reflect what the ultimate end result should be or what is realistic? Since objectives are 
to be achievable, I believe it is better to set them high enough to be challenging but low 
enough to be reachable. The next planning cycle we can always set the target a btt 
higher. Eventually we will get where we want to be. 

The step after choosing targets or selecting measurable objectives is identifying how 
you are going to accomplish your objectives. Objectives are the ends we want to achieve, 
strategies are the means. Earlier this morning, Doug Zweizig referred to the importance 
of coming up with a list of alternative activities which you could evaluate, and from 
which you could choose those most effective in meeting your target Developing such a 
list allows a library to explore many options and objectively select some. For the 
objective to reach 75% of the preschool population, most of our strategies involve 
publicity. We have a brochure called "Catch 'em in the Cradle" based on the ALA 
pregranv which has been very etfecttve. We had wonderful response to that. We gave 
it out to aU new parents at the hospital, and we found that people would come into the 
library with it in-hand asking for some of the materials listed. One of our strategies is 
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to make sure that the brochure gets an even wider distribution through doctor's offices, 
daycare centers, etc. We listed a variety of such strategies, and chose those we thought 
would best get us where we wanted to go. 

To try to pull all this together, a bit, I want to refer back to the title of my talk, 
"What is good children's library service and how can you tell?" I believe the existence 
of a plan, with goals and measurable objectives (in other words a clear understanding of 
what you want to accomplish), if implemented and monitored, can tell a library if it 
has good children's services. However this is only part of the picture. Most libraries, 
like Janesville Public Library, will have a limited number of goals and objectives 
which cover the whole realm of library service, from homebound service and reference 
to fundraising. It certainly does not include everything we do. The plan includes targets 
that stretch us, that will result in more or better service for the community. It includes 
areas that need improvement, that we want to focus on. But these targets are only a 
portion of what the Janesville library does. 

The many things that we do that are not in the plan we keep track of by listing 
them on an inventory of services. We designed the list around the roles, with services 
and materials listed under their appropriate role, even roles we have not selected. It is 
helpful to review this list periodically. Our Board and staff looked at each item on the 
list and made a conscious decision whether to maintain the status quo, increase, or 
decrease our efforts, or eliminate the service entirely. Evaluation of these services is 
much less formal than what we go through with our objectives. If we want to maintain 
our present number of volunteer microcomputer helpers, we can easily monitor that. If 
we want to gradually eliminate our filmstrip collection and build a children's video 
collection, that also is easily monitored. This inventory, or operational work-plan, 
provides informal targets. Combined with the more structured, higher impact targets in 
the library's plan, these provide the basis upon which we can determine what good 
children's library service is. 

It is essential in my library, and I believe in all libraries, that we know where we 
are going and that staff and Board have a common vision. During a discussion earlier 
this morning there was some talk about the possibility of initiating planning in the 
children's department when it is not being done elsewhere in the library. It occurs to me 
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that if my reference department had done this, developed plans and objectives they 
wanted to accomplish, it is likely their vision and mine would not be the same. As an 
administrator I believe that anytime planning and evaluation goes on within a 
children's department, it should only be with the support and common understanding of 
the administration. It is essential the children's department have the same vision as 
the organization as a whole. Resources are always scarce, whether they are human or 
financial. Tradeoffs are always made. A department planning by itself is not forced to 
consider these tradeoffs. It does little good to develop plans that cannot be 
accomplished or that require sacrifices elsewhere in the institution which the 
administration or Board are not willing to make. The Children's Department, like the 
Reference Department, is part of a larger organization and cannot function 
independently when planning is involved. 

From my point of view, good library service has to begin with a common vision 
shared by staff and Board. I don't think there is a formula we can use; there are too 
many things that enter into the equation. Following up that common vision with lofty 
goals (I say that in all sincerity) and specific measurable targets is essential before you 
can begin to evaluate the kind of children's services you are providing. 
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Introduction 

The purpose of this paper is to discuss program evaluation concepts and methods, 
with specific attention to the ways in which they can be applied to children's services. 
The special developmental, demographic, and political factors that may enhance or 
inhibit the use of program evaluation by librarians serving children will also be 
addressed, and examples of program evaluation instruments now used in several 
libraries will be appended for consideration, replication, or adaptation by interested 
readers. 

The article is not intended to be a comprehensive overview but rather a clear basic 
delineation and defense of the process with resources noted for further self-*tudy. In 
fact an attitude of self-study is intrinsic to the entire concept of evaluation; without it, 
most techniques are useless. To be an evaluator, one must care enough about what one 
does to subject it to careful scrutiny without resentment. Since many children's 
librarians are already resentful about doing important work for little money, status, or 
recognition, evaluation may be automatically suspect. The article is based on the 
premise that children's services are too important not to be evaluated! 

There are some important differences between the concept presented in this article 
and traditional concepts of evaluation in public libraries. Until the last fifteen years, 
evaluation of library services consisted almost exclusively of measuring oneself against 
national or state standards. These standards were highly prescriptive in terms of the 
number and qualifications of staff, materials, and square footage. They were generally 
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derived from peer-accepted 'test practice" in existing libraries, with some political 
compromises to make them reasonably attainable by most libraries. 1 

The problem with these prescriptive standards, beyond their obviously 
self-serving nature, became apparent in the 1960s when a group of researchers from the 
Rutgers libraiy school began looking at the differences among the "outputs" of similar 
libraries, all of which met the prescriptive "inputs." They quickly found that putting 
standard resources (i.e., staff, materials, space, etc.) into a library did not necessarily 
assure that standard activities (i.e., circulation per capita, percentage of reference 
questions filled per questions asked, program attendance per capita, etc.) would come 
out cf it. Thus the conceptualization of prior library standards was shown to be faulty, 
and the profession then started to look critically not only at what different libraries 
were doing with different resources but also at whether the libraries should adhere to 
any external prescriptive standards unrelated to local institutional objectives at all. 2 

To say that the idea of prescriptive standards died hard among children's 
librarians— especially in the public library— is a vast understatement evidenced by 
their continued publication in the mid-19803. 3 Why such resistance continues is open to 
conjecture, although in fairness it is not all inclusive. In 1985, a formal feasibility test 
of the Public library Association's (PLA) Output Measures methodology was conducted 
with a group of Wisconsin's children's librarians, 4 and a program on output measures 
was held at the ALA annual conference by the PLA Library Service to Children 
Committee. 

While the techniques described in this article may extend those measures 
promulgated by PLA and since they are not aimed at a strictly juvenile population, the 
bias of the author is definitely toward the means by which local children's services can 
be shown to be effective, rather than toward the degree to which they meet externally 
imposed piescriptions that may have nothing to do with local history, resources, or 
needs. 
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Evaluation— What It Is Not 

Since there is a pervasive tendency among youth-serving librarians to perceive 
evaluation as either an attack by a hostile administrator, or a once-and-for-all 
measure of ultimate worth, the following statements are designed to dispel mythology 
about what evaluation is or is not. 

(1) Evaluation is not the way by which one's ultimate worth is measured. In fact, 
for one seeking assurance of ultimate worth, evaluation should probably be avoided for 
mental health. That evaluation can be a distinctly threatening activity is quite 
clearly delineated by ChelimsKy. 

Wildavskv points out that "If you don't know how to make an 
™aS 

know how to evaluate, it becomes a problem for others. In fact, a major 
SSem ta toe use of evaluation has been the threat it poses. .. . First an 
evaluation report is public information which, once generated, cannot 
tetpt ^Xritafted to the private use of the decision-maker Thus, 
? prides persons other than the responsible decision-maker wtth 
information which may adversely affect that decision-maker Second, 
it is a force for change. It seeks ways to improve an existing set of 
activities, no matter what the purpose of the evaluation... 
improvement always involves change, rather than the statusquo, and 
S can appear threatening. As James Abert has put it: 'The setting 
ofprogram objectives and the choosing of evaluations are in themselves 
very emotional undertakings. Program managers generally are not 
anxious to do it. In fact, trust, confidence, honor, and many of the more 
noble aspects of life seem to be strongly challenged by evaluation, s 

(2) Evaluation is not always complicated. An example of a deceptively simple 
evaluation, easily adaptable to children's services, is the Lodestar project carried out 
by the Patrick Henry Branch of the Fairfax County (Virginia) Public Library in spring 
1985. After determining that young adults were an underserved market, the librarians 
designed a program series with a specific logo (the Lodestar) targeted to the adolescent 
age group that culminated in a contest to win a star which would then be named for the 
contest winner. Promotion of the series involved speaking to all the English teachers in 
local schools and distributing tickets through them to their students for the contest. To 
actually enter, however, the young adults had to drop off their tickets at the library. 
Thus, the objective of raising awareness of the library among an underserved group was 
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evaluated (measured) by the number of entries returned divided by the number of entry 
tickets given out. This process could easily be adapted to measure summer reading club 
promotion in specifically targeted schools. 

Admittedly, this evaluation only measured the effectiveness of the promotion, not 
the return visits of those introduced to the library this way nor the proportion of young 
adults for whom this was a first visit, but it was appropriate for what it did, and it 
was simple. Many librarians promote their programs in a total vacuum; however, in 
Fairfax County they decided how the program would be measured at the same time 
that they planned the program. 

Another example of a deceptively simple evaluation comes from the Wolfsohn 
Public Library in King of Prussia, Pennsylvania (see appendix A). When the children's 
librarians there initiated a toddler story hour, they adapted a one-page evaluation 
form which asked parents of participating toddlers not only to observe their children 
at and between story hours to record their impressions of the process of the program, but 
also to observe the program's impact on their toddlers over time. 6 This is a more 
sophisticated method because it involves nonlibrary staff, requires voluntary 
cooperation, and measures more than one aspect; it is, however, simple. 

(3) Evaluation will not always prove what you want it to. A prime example of this 
was a feasibility study conducted in 1985 by the author of taping in-hou*" 1 oicture-book 
storytime programs for rebroadcast on cable television. The study consisted of taking a 
sample of titles in present use by staff in Virginia Beach (Virginia) Public Libraries, 
mailing request letters for permission to broadcast to the publishers of these titles on a 
particular day, and then tracking the response time and the percentage of positive 
responses to determine whether one could do such a program and how long it would take 
to organize it. Since there was a significant amount of anecdotal evidence that the 
publisher did not always hold the copyright and that repeat mailings might be 
necessary, the rate of such repeat mailings was also noted. 7 The results indicated in 
table 1 show that, for this series at bast, it was possible not only to receive enough free 
broadcast permissions to have a viable program series, but also that the program could 
be set up within a four-month period. 
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One would have to replicate this process successively with different samples of 
titles and study the differences among the results before deciding whether these 
permission and response rates held true in general or just for this particular sample. In 
this instance, though, an evaluative study disproved the previously held belief that 
copyright clearance presented a serious hurdle to planning such a program. 

TABLE 1 
Cable Storytime Feasibility Study 
(see appendix B for specific titles) 



Inquiries to Publishers 

Initial letters 
Additional 
Total 

Turnaround Time 
One month 
Two months 
Three months 
Total 



24 
6 
30 



8 
8 

6 
22 



Number of Titles: 42 

Response from Publishers 

Initial responses 18 
Additional 4 
Total 22 

(73% of Publishers) 

Fees 

Free (1 time only) 
Free (3 times only) 
Free (unlimited) 
Total free 
Fee required 
Range: $25 -$500 



9 
5 
8 

22 (52% titles) 
3 (7% titles) 



(4) Evaluation is not always quantitative (U, counting things) even when the 
results are presented and analyzed numerically. The cable storytime study offers a good 
example of this concept. The actual method involved sending the same letter to 
twenty-four publishers on the same day and then tracking the responses to see what 
patterns emerged. The Center for Early Adolescence evaluated client satisfaction with 
after school programs by asking the young adolescent participants a structured series of 
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forced-choice and open-ended questions. The process was systematic and evaluative 
because they asked the same questions of participants at different sites thereby 
establishing a basis of comparison (see appendix C). 

Reference services to adults have been evaluated by means of the proxy patron 
method whereby a proxy patron asks a real reference question to see not only if one gets 
an answer but also what kind of answer. 8 This method could be used by children's 
librarians to evaluate reference service to older children and /or young adolescents. It is 
a more useful method than the "reference fill rate" of Output Measures because the 
quality of the answer can be studied as well as the interpersonal climate within which 
the answer was given. Unfortunately, this method is more difficult to administer than 
the simpler fill rate. 

(5) Evaluation does not solve problems; it only provides the evidence needed to 
solve problems. For example, an administrator who feels that a specialized children's 
services staff is inflexible and expensive, as opposed to generalists who, theoretically, 
can work equally well with all ages, may be given pause by a well-designed 
descriptive evaluation study which documents the number of adults served by the 
children's staff and the activities pursued by the children's staff when children are not 
in the library. The evidence from the study may not totally erase the bias, but it will 
possibly help point out that the problem is bias and not fact. The study might also 
prove that the director is correct about inflexibility and allow the children's staff to 
look at themselves objectively, based on facts rather than resentment. Evaluation is not 
without risk. 

Evaluation— What It Is 

Evaluation is the means by which a program, service, or activity is shown to have 
or have not worked or to examine what might work. It involves a systematic and 
explicit comparison of what actually occurred with what was planned. As a recent 
tutorial in American Libraries (October 1985 to February 1986) put it, evaluation asks 
the question, "Are we there yet?" 9 Other definitions include: 

...the process of determining whether something is what you want it to 
be.™ 
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...examining and weighing a phenomenon against some explicit or 
implicit yardstick." 

...an attempt to devise in some normal way a procedure for obtaining 
evidence or assessing how well a goal or objective... has been met* 

A systematic way to observe and describe what and how well you are 
doing to yourself and to others. . ,P 

Evaluation is an intrinsic part of program design. The methods by which progress 
will be measured must be part of the original conceptuaUzation process. One cannot 
decide after the fact that it would be nice to know if a particular activity (e.g., nursery 
school visits) had a particular result (e.g., increased interest in books) if the method by 
which the result is measured (eg* a survey of nursery school staff before and after the 
visif circulation to those chudren/fanulies/staff, etc.) is not built into the 
data'-gathering process of the program at the outset. It is especially ta^ortmt to 
evaluate programs being done for the first time to establish a baseline. This simply 
means a record against which future efforts can be compared. 

The intentions (objectives) of a program should be clearly stated and measurable 
because the presence or absence of such objectives determine not only whether a program 
can be evaluated or r jt but also what measurement is most appropriate. A good example 
is the ubiquitous summer reading club. If the stated objective is to encourage children to 
read over the summer, then the percentage increase in circulation of juvenile materials 
or the percentage increase in new juvenile card registrations during the summer months 
might be useful measures. If on the other hand the objective is to maintain reading 
skills over the summer, then such measures are imprecise. A better measure would be 
before and after assessments of reading test scores, comparing similar children who did 
and did not participate in the summer reading club. Ironically, in the evaluation of most 
summer reading programs, the second objective is intended, but it is measured by the 
first set of measures, or worse, only by documentation of increased or decreased 
participation rates without any attention to the percentage of repeaters (i.e., the effect 
of repeated participation) among those participating. 
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An evaluation is done through a series of observations which simply means "any 
method used to gather data."' 4 Frequently the observation method is of necessity some 
type of survey. An entire body of knowledge is available about survey research. One of 
the best overviews of the subject is an article by Barbara Moran in which she says: 

Taken step by step, survey research is relatively easy to do. Much help 
is available to beginning researchers from those already familiar with 
the method and from books on the topic. The first survey is the most 
difficult, but with careful planning, major errors can be avoided. Doing 
research is a lot like swimming; the only way to learn is to get in the 
waterjs 

Appropriate evaluation methods attempt to control for a variety of "threats" to 
their validity in attributing an effect to the program rather than to external forces 
outside the control of the program manager or to unintentional bias on his or her part. 
The possibility of a cumulative effect from repeated enrollment in the summer reading 
club program is an example of a factor an evaluator of that program should control for if 
by no other means than at least by documenting repeaters so they can be compared with 
first timers. 

Levels of Evaluation 

There are three distinct levels of evaluation, and it is important to understand the 
differences among them not only because they are intended to do different things, but 
also because the sophistication and resource", needed to accomplish them increase with 
each level. 

The first level is called a process evaluation. This level of evaluation measures the 
efficiency of the way in which a program is organized (i.e., the process of the program). 
In a process evaluation, the various elements of a program are reviewed., usually 
through descriptive recordkeeping. This review looks at the way in which the 
individual program elements are activated and relate to each other. A very good 
example of such an evaluation is the assessment done by the children's coordinator of 
the New Hanover County (North Carolina) Library. Using an instrument designed for 
the purpose (see appendix D), she isolated each component of the story hour process and 
rated the program giver who presented it. Besides assessing the quality of the program 



C«*"°n SCHOOL OF LIBRARY AND INFORMATION STUDIES 

56 



9 

ERIC 



61 



by examining its process, this method is extremely useful for identifying gaps in staff 
training for this program. 

Another kind of process evaluation which most children's librarians do intuitively 
is observing peak use times within the library and matching staff schedules 
accordingly. The actual documentation of these patterns has led in some libraries not 
only to more efficient staff scheduling, but also to better hours for t, children's room 
itself. This kind of ongoing process evaluation, if well documented, can show not only 
changes over time but seasonal variations within the same time period. 

Richard Windsor, an expert in evaluation research, comments that "the importance 
of specifying process evaluation procedures during the early states of program 
development and introduction cannot be overstressed.''^ The fact that this 
documentation of process was not initiated when children's services started in the late 
19th century is no excuse not to start it now especially when a few on-site researchers 
have begun to describe HP 

The second level of evaluation is called program evaluation. This term is also used 
genetically to mean all levels of evaluation, but in terms of the specific levels of 
evaluation its meaning is synonymous with impact or effectiveness. In contrast to 
merely documenting a process, this level of evaluation attempts to determine whether 
changes that have happened within a specific population at a specific location 
occurred because of the program. The Wolfsohn toddler storytime evaluation is an 
excellent example of a program evaluation design, because it attempts to determine the 
changes in behavior of the toddlers exposed to the Wolfsohn program. 

Another example of a program, as opposed to a process evaluation, would be 
determining whether attitudes toward reading changed as a result of booktalks by the 
local children's librarian in a particular classroom or in a class visit to the public 
library. With the cooperation of the teacher and school librarian, it would be fairly 
easy to use a standard attitudinal measurement, such as those included in Motivating 
Children and Young / Mts to Read,» to find out the attitudes of the students toward 
reading both before and after the booktalks. The evaluation design might be further 
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strengthened by assessing a similar group of children who did not hear booktalks and 
compare the before and after results of both groups. With a comparison group, the 
possibility that the pre-booktalk assessment influenced the post-booktalk one would 
be controlled for because both groups of similar children would have been exposed to the 
influence of the assessment but not the influence of the booktalks. 

This is essentially the process followed by Joni Bodart, author of Booktalkf 19 and 
Booktalkl 2 20 in her dissertation research. One does not, however, need to do a 
dissertation to implement this kind of evaluation as long as school officials are 
cooperative. 

Measuring a change in attitudes toward reading as a result of booktalks implies 
that this is the intended impact of the program. If another impact is intended— such as 
increased circulation by the children who heard the booktalk— another kind of 
observation method is required. A mere increase in the circulation of the itles used in 
the booktalk may be too imprecise because it would disguise the effects of peer 
interaction among children who were exposed to the booktalks and the friends with 
whom they talked who were not. If the intent of the booktalk program is to promote 
reading among children or to call attention to "sleepers," a circulation increase in the 
titles used in the booktalk would be an appropriate measure. In a program evaluation, 
the impact of the program and possible effects on it must be sorted out at the time the 
evaluation design and data gathering are being determined 

A program evaluation attempts to ascertain the presence or absence of a variety of 
"threats" to the possible effectiveness of a program. Many of these problems can be 
controlled for by the program administrator from the beginning, or, if not controlled for, 
they can be examined retrospectively to explain a lack of impact unfairly ascribed to 
the program. 

The most familiar threat is called history in evaluation jargon. This is the 
unexpected snowstorm that keeps an audience away on what otherwise seemed to be an 
ideal night for an evening holiday story hour, or the television movie that results in 
indifference to an outstanding booktalk. Essentially, history is any extraneous event 



Chelton 



SCHOOL OF LIBRARY AND INFORMATION STUDIES 



over which the program planner has little control and whicn interferes with the 
impact of the program. Of all the threats, history is the hardest to control but the 
easiest to explain after the fact. There is probably not a children's librarian working 
who has not already experienced it. 

A second threat is called maturation, which means simply that the program is 
being evaluated while it is still immature enough that the "kinks" of immaturity (i.e., 
inexperienced staff or participants) are unduly influencing possible effectiveness. An 
example might be something as simple as a drop in attendance in the first weeks after a 
new librarian takes over a storytime program. This does not mean that the program is a 
failure, only that it needs an initial adjustment period. If the lowered attendance 
persists however, it is more likely that something inherent in the program itself (i.e., 
the titles chosen, the length, or the librarian's skill) is a problem unless something has 
changed in the target community (e.g., massive layoffs among parents, a gasoline crisis 
or transportation strike, ethnic/socioeconomic shifts, etc.) The latter would be another 
example of history. The threats cannot be examined in isolation from one another, and 
in this case they may be interacting. 

The third threat is called testing. This occurs when the means of observation 
-whether an interview, a survey, or an attitudinal assessment such as the one 
mentioned earlier in connection with booktalks-influences the reaction of the program 
participants. If children are asked about whether they like to read, then a librarian 
comes to talk to them about books to read, and they are then asked again whether they 
like to read, the children may well deduce that they should say "yes" to this person. 
Thus the second responses will be unduly influenced by being asked the question 
previously, and the librarian will never know the real effect of booktalks on reading 
attitudes, or worse, assume a vastly inflated role in changing them. The best way to 
control for this threat is by using a similar group, also exposed to the before and after 
influence of the observation method but not to the program. It is also important that the 
comparison group of children not be able to interact with the ones experiencing the 
booktalks, or the distinction of the two groups is lost *.s a basis of comparison. 
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A threat related to testing is called instrumentation. This means the degree to 
which the method of observation is valid and reliable. Reliability is the extent to 
which the same measure gives the same results on repeated application. The influence 
of instrumentation is shown in national polls on public policy issues— such as 
abortion— where the answers vary depending on how the question is phrased. The key 
to reliability is whether the particular phrasing is similarly interpreted by all the 
people responding. This problem is possibly exacerbated with children because of their 
varying levels of cognitive development even among children of the same age. When 
the method of observation is a test or survey, it is extremely important to try it out on as 
many children as possible— excluding the actual audience to be surveyed or tested— to 
make sure that they interpret the question in the same way. For very young children, 
these methods of observation are not appropriate. 

Validity is the extent to which a criterion really measures what it says it does. A 
measure can be reliable without necessarily being valid. George iyElia has pointed out, 
for example, that the output measures of materials availability, proposed by the 
Public Library Association, may really be measuring user behavior rather than 
materials availability and are invalid. 2 From a children's services perspective, all of 
the first-level output measures are invalid because they do not take the age structure of 
the population into account. Not only do they ignore the proportion of children in the 
population but also the proportion of adults in the child rearing years who might be 
using the library for their children. At this point the PLA's output measures obscure 
children's services more than reveal them. The validity of these measures will 
probably be debated in professional literature for some time, and children's librarians 
would be wise not to let the debate go on without them. 

One of the ways in which one controls for the threat of instrumentation, besides 
scrupulous pretesting with members of the intended audience, is by inviting outside 
experts and/ or peers to comment on the validity of the measures chosen to document a 
particular activity. There are also statistical methods to assess validity and 
reliability. An excellent discussion of all the issues involved in good instrumentation 
appears in the chapter on data gathering in Research for Decision-Making 71 as well as 
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in Windsor's book, and in the titles cited in Moran's article. These methods are outside 
the scope of the present article. 

Another threat is called regression. This refers to a statistical phenomenon where 
by subjects chosen as an extreme example of any phenomenon will "regress" over time 
toward the average example of that phenomenon. Regression makes it impossible to 
tell whether the changed phenomenon was caused by the program or the types of 
people observed. The regression threat is related to another called selection. This 
refers to the group chosen for study and how representative they actually are of the 
entire population about whom the investigator might want to draw some conclusions. 
Selection also refers to characteristics of groups chosen for comparison with the group 
receiving the program. Selecting the wrong group of people in the first place, or the 
wrong group for comparison, can interfere with determining the impact of a program. 
The same references cited for an expanded study of instrumentation also include 
information on sampling and selection. 

Even when theselection process has been sound, attrition can pose a threat. This 
refers to a significant loss of program participants and may give a clear signal that 
something is wrong with the program. Attrition can happen for a variety of reasons: 
the program was too long, the publicity was misleading, etc. Attrition, while 
dismaying, is very important, especially when those who drop out or leave a program 
differ markedly from those who stay. A common mistake is to overlook these 
differences and ascribe success to the program because a reasonable number of people 
stayed with it, especially when an examination of differences might reveal that the 
program was least successful with those it most intended to reach. Summer reading club 
attrition by age and sex begs for such an analysis. 

While all the threats to program effectiveness can be discussed separately, they 
tend to interact in real situations. A thorough understanding of the individual threats 
to a program provides the necessary insight not only to make programs mo/e responsive 
in the first place but also to analyze in a systematic way the program's actual effects. 
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There is a third level of evaluation called evaluation research which is usually 
beyond the resovrces and training of practitioners, but the results of this level of 
research should, oe known to them. Evaluation research is the process by which the 
theories underlying practitioner programs are scientifically tested. Bodart tested the 
hypothesis that booktalks improved the reading attitudes of adolescents. 23 Smardo 
looked at the effect of different types of story hour presentations on the receptive 
language of children. 24 Powell et al. investigated the relationship between certain 
childhood experiences and adult library use.^Heyns studied summer activity that 
influenced children's vocabulary scores. 26 Greene compared three different types of 
library-based early childhood centers. 27 Fasick and England compared media 
preferences between childhood users and nonusers of a Canadian public library. 28 

Unfortunately, most academic research related to children's services has focused on 
the contents of materials produced for children rather than the impact of library 
services on them. Benne indicated that measurement of children's services was a 
problem because of a lack of clearly defined goals, 29 and the author suspects that this 
problem carries over to the conceptualization of research. It is doubly important that 
the little research that has been done be familiar to practicing children's librarians so 
that they understand that their programs are based on sound theory rath * than on 
tradition. Knowledge of evaluation research can enhance program justification and 
planning. 

Politics of Evaluation 

The fact that a program has clear measurable objectives, valid measures, and 
sufficient resources to document itself does not ensure a successful evaluation although 
all those factors must be present in order to do one. Evaluation serves more than one 
purpose within an organization. Besides providing evidence for decision-making — and 
possibly proving accountability to the public-^an evaluation should also gain 
administrative support for solving the problems it reveals. Thus the ultimate success of 
an evaluation depends on "the skill of the evaluators in understanding the value 
systems present in the organization." 30 If the library does not have a capacity for 
self-criticism and change, an evaluation may only be an exercise in futility. Davis and 
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Salasin have proposed eight factors to examine and consider before beginning an 
evaluation process: 

Ability to undertake and use an evaluation in terms of notary 

resources, available staff, and staff energy to mount both the study 
and for the implementation of resultant recommendations. 

Vaht» hdd Jn theorganization. Specifically, is the organization ^secure 
into ^environment, open in its <^™-^^ * 
change, and is management able to accept criticism and act on it?... 
Information available or that can be procured to support the 

Ciix^tenS prevailing at the time of the evaluation. Elements 

examined include recent changes in programs, new or o d 

Ship, stete of client relationships, internal conflict, openness 

in the exchanee of ideas and criticism. 
Timmg ot*e equation should coincide with other activities or 

oroerams that might encourage change.... 
ObhSTo changed there is dissatisfaction with the status quo, 

changes can occur more easily and naturally. 
Resistance to change is always present in the organization, and an 

StenSof its sources and strengths is critical for anyone 

evaluating the organization's programs. 
YieW. From the outset, there must be assurances by^emen that it 

considers the evaluation to be important, that the results of the 

evaluation will justify its costs, and that it expects to use the 

results to bring about desired changes3i 

While these factors are intended to enlighten an outsider who comes in to evaluate 
an organization, they are equally important for an insider to consider. Just as a 
children's librarian might feel attacked if suddenly asked to evaluate his or her 
program, a library director will feel just as attacked by a good evaluation of a 
children's program that demands change on the part of management There are 
political ramifications to "describing oneself to other," through a program evaluation, 
and they are ignored at one's peril. In many cases it may be necessary to postpone 
reporting the evaluation results until a change in management precipitates a change in 
organizational values. 
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Evaluation Problems Unique to Children's Services 
Organizational Isolation 

Most children's librarians positions are entry-level, which usually means that 
there are several hierarchical layers of authority between children's librarians and 
top management. The resources and support necessary not only to do, but to respond to 
the results of a program evaluation can be waylaid at any of the levels. This is 
particularly true in libraries converting to automation. If the children's librarian 
evaluates the use of the catalog by children and presents children's needs for multiple, 
popular subject access points before the implementation of automation, he or she may be 
particularly unwelcome to a branch librarian or childre n's coordinator who has 
received a clear message from the top that funding will never cover that level of 
quality in the system. 

On the other hand, if in the course of a post-implementation evaluation the 
children's librarian points out problems, the middle manager may "kill" the message so 
as not to disturb the uneasy complaisance of the decision-maker, and the children's 
librarian will fall into disfavor for bringing bad news. Unless a library is particularly 
open to staff suggestions for change from entry-level people, a children's librarian 
would do well to analyze and cultivate the intervening administrative personnel to 
ensure their understanding and support before embarking on any serious evaluation 
which may ultimately demand something of them. 

Geographical isolation is also a problem if the children's librarian is relegated 
exclusively to a children's room with little interaction with the rest of the staff. 32 Such 
an arrangement can literally make children's services invisible, and initial attempts to 
call them to someone else's attention may cimply be interpreted as annoying 
self-promotion. Unfortunately, in the absence of complaints from the public, many adult 
services librarians and directors are as happy to have children's services (and 
children) remain invisible as children's librarians are to have the illusion of autonomy. 
This mutually self-serving isolation is not oruy shortsighted but also does not advance 
the cause of gaining institutional support for an evaluation or for the changes it may 
indicate are needed. 
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The Age Structure of the Population 

Even though the sheer numbers of the "baby boomers" and their delayed fertility 
indicate a potential "boomlet" on the horizon, the overall proportion of children in the 
population is declining because the "boomers" are having far fewer children than their 
parents * It is equally important to understand that the age structure of the various 
ethnic groups within an overall population may be very different not only from the 
total population but from each other because of different fertility rates. Urns, in a 
community with white, black, and Hispanic populations, the proportions of children in 
each group might be respectively 15 percent, 23 percent, and 30 percent. Since cultural 
and socio-economic factors influence librarv use, it would be illusory for a children's 
librarian to credit her program with success based on participation if all participants 
are white and the changing demographics of the target population are ignored. On the 
other hand, it may be difficult to gather support for the resources needed to deliver and 
evaluate services to minority children when an aging white population comprises the 
power structure that supports the library. 

Age structure is also an important consideration in the use of PLA's level one output 
measures, since they are based solely on the total population. In a library that records 
50 percent of its circulation from children's materials in a population consisting of only 
25 percent children, an overall per capita measure totally blurs reality. 

To get an accurate count of the juvenile population of a community and its proportion 
compared to other age groups, there are now some helpful tools besides the standard 
single-year breakdowns in ^General Characteristics of the Population published by 
the VS. Bureau of the Census ."One of the most helpful is the Donnelly Demographics 
database available on DIALOG. 35 Donnelly's also publishes Market Profile Analysis* 
which lists the percentage of the youth population by census tract. 

Developmental Level of the Children 

It is somewhat difficult to ask a preschooler how or why he/she enjoyed a story 
hour program when the child has not yet developed either the thought processes or the 
language to answer. Also, children are in a power relationship with adults, and might 
easily just give answers which they feel please the adult rather than their actual 
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reaction. This means that children's librarians have to depend more on observational 
methods for evaluation rather than using methods— such as questionnaires— common 
with adult audiences who jel free to speak for themselves. 

This is not an insurmountable problem and in fact could be considered a strength 
because of the necessity of observing behaviors as opposed to accepting verbalizations 
totally unrelated to behavior. The key to this problem is to find consistent, unobtrusive 
observers who can rate behavior which is unambiguously described, such as "The child 
sits quietly for ten minutes during storytime/' or 'The child listens attentively for at 
least five minutes." If the observers are well trained and the behaviors unambiguously 
described on an instrument that has been rigorously pretested, the lack of young 
children being able to speak for themselves does not present a great problem in 
evaluating programs for them. One of the means of ensuring a good evaluation of this 
type is to have the several observers rate the same child with the same instrument at 
the same time to rules out any effect of observer variation before Jte program actually 
begins. This particular check on observer variation can also be repeated at different 
times during the program. Given the need for these methods and the importance of the 
services to young children, it is frustrating that so few public library children's rooms 
do not incorporate two-way mirrors into the architecture. 

Another technique that can be used with children comes from con r u tier marketing 
called focus interviews. In this method, a group is chosen and then asked specific 
structured questions by an adult facilitator. The entire process is videotaped for later 
analysis. This technique is useful with verbal children for whom reading or writing 
may still be a problem. The issue is to find an evaluation method which is appropriate, 
useful, and affordable, not just to copy a method more appropriate for adults or 
adolescents, or worse, to declare services to young children impossible to be evaluated. 

Parental Involvement 

While there may still be some children's librarians who regard parents as 
interlopers into their professional domain, it is hoped that most children's librarians 
welcome parental involvement in the process of introducing children to the reading 
experience. In terms of evaluation, parents can perform several invaluable functions. 
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They can give consent for their children to be observed or interviewed where necessary, 
and they can serve as observers themselves to document behavioral changes in their 
children as the result of library programs. Parents can also provide advice at the point 
at which an evaluation is being designed even to the point of helping to pretest 
instruments with their own children. 

It is important, when involving parents in an evaluation, to make it clear to them 
that the existence of the program does not depend on the evaluation, or they may give 
biased responses. Unfortunately, the concept of evaluation as self-. :udy rather than as 
proof of worth is as misunderstood by the general public as it is by librarians. 

Working parents can be involved by mail or telephone, or the child's day-care 
provider or nursery school teacher might be called upon for help. Given the 
multiplicity of child care arrangements among working parents and the lack of 
credentials among providers* this may be very frustrating, although to be pragmatic, 
these individuals may be the only adults actively involved with a particular child 
who can be called upon for help. The reality is that children do not control their own 
lives-the adults who care for them do-so the adults must be involved in evaluation 
efforts. 



Seasonality 

While it is appropriate to measure a discrete event or series at the time it takes 
place, it is not appropriate to measure ongoing services with significant seasonal 
variations only on an annual basis. Unfortunately, this approach is recommended in 
Output Measures. In services as distinctly seasonal as those to children, this process is 
meaningless. In fact, the Wisconsin feasibility study of using output measures alludes to 
this problem. While libraries hardly need to return to the days of mindless daily 
manual record-keeping, it seems reasonable that children's services be measured 
quarterly at the very least. 

Interagency Links 

The same children are shared by and may be exposed to similar activities in the 
public library, the school library, the classroom, the day-care center, the nursery 
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school, and the home. While this continuum of si tared influence potentially can 
interact to the benefit of the child, these linkages make it very difficult to evaluate 
the effect of an individual program. A prime example was raised by Benne: 

The measurements used may also hamper the efforts of the children's 
librarian to enter into a cooperative relationship with the schools. If 
the librarian visits classrooms to tell stories or give booktalks, is the 
effectiveness of the activity to be measured by the actual presence of 
children at the public library within a stated interval of time? If the 
result is to increase the child's interest in ideas and in the use of 
materials in the classroom or school library, does this further the goal 
of public library service?38 

One of the ways to control for interactive efforts is to plan for them in setting 
program objectives against which any effects in any of the places will be observed. Do 
not automatically assume that the results will be bound to one institution. The other 
way to control for interactive effects is to limit history in the other agencies through 
their initial involvement in planning the evaluation. 

Conclusion 

While one might lament the lack of systematic evaluation of children's services in 
public libraries in the past, the changing demographics of the family, 39 the changing 
proportion of children in the population, 40 the changing nature of children's services 
themselves, 41 and various children's services staffing arrangements (i.e., generalist, 
paraprofessional, etc) beg for an analytical examination. For those daunted by 
political, conceptual, or resource constraints, the motto of the Children's Defense Fund 
comes to mind: "Dear Lord, be good to me. The sea is so wide and my boat is so small." 42 
Evaluation can help keep the boat on course. 
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Appendix A 

Wolfsohn Memorial Library, Pennsylvania 
Toddler Story Hour 
Parent's Evaluation 

We would appreciate your comments about this program in order to help evaluate its 
worth and to help determine whether it should be continued. 



Time of day: 
^ength: 

(each program) 
Length: (series) 
Place: 

Size of group: 



Too early. 



Too late 



OK 



Too short__ 
Too short__ 
Too small_ 
Too large. 



Too long. 
Too long. 



Too many distractions. 
OK 



OK_ 
OK_ 
OK 



Program and materials used: Not enough planned r Too muchplanned 



Child not interested in stories.. 

Child not interested in activities 

Stories, activities too old for child 

Stories, activities too young for child. 
Stories, activities OK 



Would you attend this program again? 
Why or why not? 




Since you both began participating in the program, hav 

your child: 

Longer attention span 

Greater interest in looking at books 

Greater interest in listening to stories at home 

Greater enjoyment and interest in coming to 

the library 
Greater rapport with other children 
Greater rapport with adults outside the family 



noticed any changes in 



No 


>es 


No 


Yes 


No 


Yes 


No 


Yes 


No 


Yes 


No 



Do you have any comments you would like to add?_ 



Source: CY.P.S.L's Idea Exchange Handbook, Pennsylvania Library Association, 1981 
Reprinted with permission of Wolfsohn Memorial Library. 
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Appendix 8 
Virginia Beach Public Library 
Cable Storytime Feasibility Study (Publishers/Titles) 



Abingdon Press 
Atheneum Publishers 
Children's Press 



Curtis Brown, LTD 
Doubleday & Co., Inc. 
Greenwillow Books & 
Lothrop, Lee & Shepard Books 



Harper k Row 

Holiday House, Inc. 

Holt, Rinehart and Winston 

Houghton Mifflin Co. 
Little, Brown & Co. 

Macmillan Publishing Co., Inc. 



Pantheon Books 



HUMBUG RABBIT 

MAY WE SLEEP HERE TONIGHT? 

BLUE BUG GOES TO THE LIBRARY 

BLUE BUG'S VEGETABLE G ARDEN 

TIME TO RHYME WITH CALICO CAT 

PUPPY TOO SMALL 

8,000 STONES: A CHINESE FOLKTALE 

GOOD AS NEW 

ONE DUCK ANOTHER DUCK 

THE SURPRISE 

PEA SOUP & SEA SERPENTS 

THE GREAT BIG ESPECIALLY BEAUTIFUL 

EASTER EGG 
GIVING TREE 

MOTHER RABBIT'S SON TOM 

THE BIG BUNNY AND THE EASTER EGGS 

BOO WHO? 

LIANG AND THE MAGIC PAINT BRUSH 
YUMMERS 

MARY ALICE, OPERATOR 9 

ANDREW'S BATH 

TWO GREEDY BEARS 

ASK MR. BEAR 

MUSHROOMS IN THE RAIN 

CORNELIUS 

SWIMMY 

THE BIGGEST HOUSE IN THE WORLD 
WINTER PICNIC 
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Appendix B 
Virginia Beach Public Library 
Cable Storytime Feasibility Study (PublishersflTitles) 

(continued) 



Prentice-Hall, Inc. 



The Putnam Publishing Group 



Rae John Publishers 
Vanguard Press 

Western Publishing Co., Inc. 

World Free Flight Press 



THE CATERPILLAR AND THE POLLIWOG 
LITTLE PEEP 
SOPHIE AND JACK 
WHERE'S SPOT 

THE VERY HUNGRY CATERPILLAR 

THE LITTLE RABBIT WHO WANTED RED WINGS 

THE LITTLE RED HEN 

THE MAGIC BOAT AND OTHER CHINESE FOLK 
STORIES 

HOME FOR A BUNNY 
THE GOLDEN EGG BT 
THE VERY HUNGRY CATERPILLAR 
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Appendix C 
Center for Early Adolescence 
Program Participant Questionnaire 



We are interested in knowing what you think about this program. Please take the time 
to answer these questions so we can make our program even better. 

1 . How old are you? 

2. Are you male? female? 

3. How often do you come to this program? 

4 What do you like about this program? 

5. What do you not like about this program? 

6. Is there an adult here whom you talk to when you 
want advice or just want to talk about personal 

concerns and problems? yes no 

7. Do you think this statement is true or false? 
"Almost everyone at this program has a 
close relationship with at least one adult 

staff member/' true false 

8. Do you think this is true or false? "The 

adults at this program really care about me." true f alse 

9. What are the three most important rules here? 

10. Do people frequently break the rules here? yes no 

Why is that? 

What happens when they do? 
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11, Do you have a voice in making decisions and 
planning activities here? 

12. Do you feel, safe here? • ^ 

Explain 



yes no 

no 



yes no 



yes no 



13. Do you get to do something you are good at here? — 
If yes, what? it no, why not? 

14. Do you get to do things you like to do here? — 
If yes, name two things: 

If no, why not? 

What could be done to give you more opportunities to do the things you like? 

15. What is the purpose of this program? What does it stand for? 

16. What changes would you like to see at this program? 

Is there a way you can help make the:« changes? yes "> 

If yes, how? 

17 What do vou think is the biggest problem young people your age have? Does this 
1 ^& your Wends deal with that problem? Hyes,how? Ifno,why 
not? What could be done here to help you with that problem? 



18 



How do you get here after school (for instance, by bus, bike, walking, carpooD? 



19. Is there something else you would rather be ^ 
doing after school? 
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20. On days when you do not come to this program, what do you do? 
(Check all that you do.) 



a . take care of young brothers and sisters yes no 

b. participate in school activities (such 

as cheerleading, clubs, and sports) yes no 

c. receive tutoring yes no 

d. participate in other organized groups 

Oi' clubs yes n o 

e. do volunteer work (such as helping in 

a hospital, tutoring) yes no 

f. do work for pay yes no 

g. participate in a non-school-sponsored 

team sports yes no 

h. play outdoors in the neighborhood, at a 

sports field, or on a local playground yes no 

i. go to a shopping district or mall yes no 

j. go to the library yes n o 

k. visit a museum yes no 

1. go to church or synagogue activities yes no 

m. hang around 

(where? ) yes no 

n. stay at home yes no 

o. visit a friend yes no 

p. other yes no 

21 . How did you learn about this program? 



22. List some things you would like to do or learn about; for example, "tour a TV station" 
or "learn how to cook." 

a. 
b. 
c. 
d. 
e. 
f. 

23. What else would you like to tell us about how you feel about this program? 

Reprinted with permission from the 
Center for Early Adolescence 
Suite 223, Cart Mill Mall 
Carrboro,NC 27510 
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Appendix D 

Youth Services Department, New Hanover County Public Library 
Program Evaluation Form 



Scale: 2 = poor 
4 = fair 
6 ■ average 
8 = very good 
10 = exceptional 



Name_ 
Date 



Program 



Rating Comments 



1. Attitude toward children 
(cheerfulness, welcoming, comfortable) 

2. Attitude towards parents/teachers 
(approachable, comfortable, helpful) 

3. Preparation 

(familiar with all material) 

4. Theme 

(appropriate, used throughout program) 

5. Program flow 

(smooth, orderly, keeps children involved) 

6. Selection of material 

(app. to age and dev. level of children) 

7. Balance of formats 

(use of books, AV and activities) 

8. Presentation 

(voice, movement, body language) 

9. Use of activities 

(as stimulation or calming factor) 

10. Control 

(of children and adults) 



Reprinted with permission of Youth Services Coordinator, 
New Hanover County Public Library, Wilmington, NC 



UNIVERSITY OF WISCONSIN— MADISON 



References 



'American Library Association. Public Library Association. Subcommittee on Standards 
for Children's Services. Standards for Children's Services in Public Libraries. 
Chicago: ALA, 1964. 

2 DeProspo, Ernest, et al. Performance Measures for Public Libraries. Chicago: ALA, 1973. 

3 New York Library Association. Youth Services Section. Standards for Youth Services 
>n Public Libraries of New York State. New York: NYLA, 1984. 

4 Zweizig, Douglas, et al. Output Measures for Children's Services in Wisconsin Public 
Libraries: A Pilot Project— 1984-85. Madison: Wisconsin Division for Library 
Services, 1985. 

K^helimsky, Eleanor. Old Patterns and New Directions in Program Evaluation. In 
Program Evaluation: Patterns and Directions, edited by Eleanor Chelimsky. 
Washington, D.C.: ASPA, 1985. 

6 Kewish, Nancy. South Euclid's Pilot Project for Two-Year-Olds and Parents. School 
Library Journal 25 (March 1979): 93-98. 

TWendling, Nancy. Porter Makes ft Cable Connection. School Library Journal 29 (April 
1983): 19-22. 

Rowley, Terence, and Childers, Thomas. Information Service in Public Libraries: Two 
Studies. Metuchen, N.J.: Scarecrow, 1971; and Gers, Ralph, and Seward, Lillie J. 
Improving Reference Performance: Results of a Statewide Study. Library Journal 110 
(Nov 1985): 32-35. 

'Robbins-Carter, Jane, and Zweizig, Douglas L. Are We There Yet? Evaluating Library 
Collections, Reference Services, Programs, and Personnel. American Libraries 16-17 
(Oct. 1985-February 1986): 624-27, 724-27, 780-84, 32-36, 108-112. 

10 Lynch, Mary Jo. Measurement of Public Library Activity: The Search for Practical 
Methods. Wilson Library Bulletin57 (Jan. 1983): 388-93. 

"Weiss, Carol H. Evaluation Research: Methods of Assessing Program Effectiveness. 
Englewood Cliffs, N.J.: Prentice-Hall, 1972. 

12 Rayward, W. Boyd. Programming in Public Libraries: Qualitative Evaluation. Public 
Libraries 24 (Spring 1985): 24-27. 

13 Chelton, Mary K. Developmental^ Based Performance Measures for Young AcUilt 
Services. Top of the NewsAl (Fall 1984): 39-51. 



Chelton SCHOOL OF LIBRARY AND INFORMATION STUDIES 

76 



ERIC Si 



"Windsor, Richard A., et aL Evaluation of Health Promotion and Education Programs. 

Palo Alto, Calif.: Mayfield, 1984. 
«Moran, Barbara B. Survey Research for Librarians. Southeastern Librarian 35 (Fall 

1985): 78-81. 

Windsor, Evaluation of Health Promotion and Education Programs. 

iTHektoen Faith The Connecticut Research Documentation Project. School Library 

Information Requests. School Library Journal 26 (April 1980). 27, and 
H*™« Craighton. Measuring Children's ^Library Collections Through 
Objective Data. Top of the NewsA2 (Spnng 1986): 309-13. 

imtomas, James L, and Loring, Ruth M., eds. Motivating Mdren and Young Adults 
to Read. Phoenix, Ariz.: Oryx Press, 1983. 

UBodart, Joni. Booktalk! New York: H.W. Wilson, 1980. 

20 . Booktalk! 2. New York: H.W. Wilson, 1985. 

«D'Elia, George. Materials Availability Fills Rates-Useful Measures of Library 
Performance? Public Libraries!* (Fall 1985): 106-10. 

Swisher, Robert, and McClure, Charles R. Research for Decision-Making. Chicago: 
ALA, 1980. 

23Bodard, Joni. Book You! Voice of Youth Advocates^ (April 1986): 22-23. 

24Smardo, Fiances. What Research Tells Us About Storyhours and Receptive Language. 
dEk Dallas Public Library and North Texas State University, 1982. 

25p OW ell, Ronald R., et al. Childhood Socialization: : Itj Effecton Adult Library Use 
and Adult Reading. Library Quarterly *A (July 1984): 245-64. 

26 He yns, Barbara. Summer Learning and the Effects of Schooling. New York: Academic 
Press, 1978. 

^Greene, Ellin. Early Childhood Centers; Three Models. School Library Journal 30 
(Feb. 1984): 21-27. 

28Fasick, Adele, and England, Claire. Children Using Media. Riding and Viewing 
rreferenZ Among the Users and Non-Users of the Regina Public Library. 
Saskatchewan: Regina Public Library, 1977. 



Chelton 

UMivERsrrv of Wisconsin— madison 

77 



erJc S j 



^enne, Mae. Educational and Recreational Services of the Public Library for 
Children. Library Quarterly AS (Oct. 1978): 499-510, 



Williams, Delmus E., and Racine, Drew. Planning for Evaluation: The Concept of 
Pre-Evaluation. LAMA Newsleiter 11 (Sept. 1985): 73-75. 

31 Davis, Howard R., and Salasin, Susan E. The Utilization of Evaluation quoted in 
Williams, and Drew, Planning for Evaluation, pp. 73-75. 

32 Benne, Mae M. Information Services in Central Children's Libraries. School Library 
Journal 26 (April 1980): 25. 

•^ouvier, Leon. itoCTi'ca's Baby Boom Generation: The Fateful Bulge (Population 
Bulletin 35:1). Washington, D.C.: Population Reference Bureau, 1980. 

^US. Bureau of the Census. Characteristics of the Population (Series P-20). 
Washington, D.C.: USGPO, 1982. 

35DIALOG. DIALOG Information Services, Inc., 3460 Hillview Avenue, Palo Alto, CA 
94304. 

^Donnelley Marketing Information Services. Market Profile Analysis. Stamford, Conn.: 
Donnelley Marketing Information Services. 

Hewlett, Sylvia. A Lesser Life: The Myth of Women's Liberation. New York: Morrow, 
1986. 

38 Benne, Information Services in Central Children's Libraries, p. 25. 

^Thornton, Arland, and Freedman, Deborah. The Changing American Family 

(Population Bulletin 38: 4). Washington, D.C.: Population Reference Bureau, i983; 
anil Edmondson, Brad. Education of Children. American Demographics 8 (Feb. 
1986): 27-29,51-52,54. 

Population Reference Bureau Staff. U.S. Population: Where We Are; Where We're Going 
(Population Bulletin 37: 2). Washington, D.C: Population Reference Bureau, 1982. 

41 Willett, Holly. Current Issues in Public Library Services for Children. Public 

Libraries 24 (Winter 1985): 137-40; Hennepin County Library. Services to Children 
in Groups Task Force Report. Ridgedale, Minn.: Hennepin County Library, 1985; end 
Denniston, Susan. Extending Library Services to Children: Reaching Out to 
Preschools and Others Day Care Facilities. Emergency Librarian 13 (Jan. Feb. 1986): 
19-20. 

^Children's Defense Fund, 122 C. Street, N.W., Washington, D.C. 20001. 



SCHOOL OF LIBRARY AND INFORMATION STUDIES 



ERIC 



Chelton 
78 



Looking Around 



Charles R. McClure, et ah 
Excerpted by permission from 
Planning & Role Setting for Public Libraries; 
A Manual of Options and Procedures. Chicago: ALA, 1987. 

Locking Around, the process of collecting mformation about the library and its 
community, is one of the most interesting phases in the planning process. It is a process of 
discovery, of finding answers to such basic questions as: What kind of community does 
the library serve? Who lives there? What do residents expect from the library? 
Information about the community and its residents helps you understand what services 
may be needed. Information about the library helps you understand how well the 
library is doing its job. Taken collectively, this information assists the planning process 
by: 

• Identifying factors in the library's environment that may affect the 
provision of services 

• Revealing community needs for library services 

• Demonstrating to community decision makers the library' s 
understanding of its services and its community 

• Suggesting possible library roles, goals, and objectives. 

This phase in the planning process is closely linked with the next, Developing 
Roles and Mission (Chapter 4). In fact, you may reverse the order of these two phases. 
Placing the Looking Around phase first helps the planning committee consider library 
and community conditions as they select roles. Developing Roles and Mission first can 
help the planning committee focus its Looking Around activities. Either approach is 
workable. 
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Looking Around has six basic steps: 

1. Determine level of effort 

2. Prepare for Looking Around 

3. Decide what information is needed 

4. Gather the information 

5. Study the information 

6. Report the results. 

Step 1: Determine Level of Effort 

Level of effort for Looking Around is directly proportional to the amount of 
information the planning committee decides to gather and the sources used to obtain 
that information. Therefore, the planning committee's "information appetite" must be 
tailored to match the library's staff time and financial resources. Three factors are 
important: 

• The time available for this planning phase: some information is 
readily available; other information may take longer to assemble. 

• The diversity present in the library's community: a relatively 
homogeneous community can be described more easily than a 
complex community 

• The rate of change in the library's community: rapid change may 
indicate a need for more comprehensive information. 

At basic and moderate levels of effort, the planning committee selects only a small 
set of data (see Step 3) and confines its information gathering to existing sources of 
information (see Step 4). At an extensive level of effort, the planning committee 
conducts a little more thorough analysis of the library's environment and may use data 
collection methods beyond the scope of this manual, such as surveying. The citations at 
the end of this chapter can provide guidance for these met f xis. 

(For a fuller presentation on levels of effort, see "Level of Effort for Looking 
Around" at the end of this chapter.] 
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Step 2: Prepare for Looking Around 

The planning committee and library staff both have an important role during 
Looking Around. Generally, the staff is responsible for gathering and assembling the 
information required by the planning committee. The committee determines what 
information it needs and studies the information collected. This step prepares the 
planning committee and staff for these activities. 

Assign Staff Responsibilities 

The library director determines who will be responsible for gathering information 
during Lc .king Around. Usually one staff member is given the overall responsibility to 
gather, organize, and analyze the information requested by the planning committee. If 
possible, this person should have experience in reference or public services, be a good 
communicator, and a good organizer. Others on the library staff assist a* needed. 
Ideally, staff assignments are made before the planning committee decides what 
information to collect. This allows those responsible, by working with the committee, 
to better understand what the committee needs and how the information r dght be used. 
If your library has a microcomputer, the use of word processing, database management, 
and spreadsheet software can make writing reports and analyzing and presenting 
information easier. 



Prepare the Planning Committee 

Two reviews can help the planning committee approach Looking Around more 
productively. First, review the basic guidelines for collecting information presented in 
Figure 1. These tips can help both librarians and planning committee members 
approach Looking Around in a positive, realistic framework. Second, review with the 
committee the range of information that might be considered. In so, doing, consider 
information about the library's community (Looking Around Outside) and information 
about the library itself (Looking Around Inside). 

LOOKING AROUND OUTSIDE 

Communities and their residents differ. These differences translate into service 
demands on the library. For example, an actively growing community with a diverse, 
highly educated population will place different demands on the library than a 
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community whose industries are declining and whose residents are predominantly 
unemployed blue-collar workers. The categories below suggest types of information 
that the planning committee might obtain. 



• Demographics: the age, sex, racial, ethnic, educational, and income levels of 
residents; family characteristics, such as single heads of household and family 
size; and community characteristics, such as the birth, death and unemployment 
rates. 

• Economic condition : the mix of business, farming, manufacturing, and service 
sectors; tax base; and economic trends—growth, decline, and boom/bust cycles. 

• Social conditions: educational and cultural institutions, clubs, societies, interest 
groups, religious groups, traditions, and community history. 

• Informational and educational services: other nearby libraries; primary, secondary, 
and post-secondary educational institutions and nontraditional learning programs; 
newspapers and magazines; fee-based information services, bookstores, radio 
stations, and cable television. 

LOOKING AROUND INSIDE 

Looking Around Inside examines the library's current condition. All aspects of 
library services are fair game: building, collections, services, staff, management 
p. grams, and budget allocation. Libraries with multiples service outlets may study 
them individually or collectively and may evaluate communication and service 
patterns among the branches and central library. The categories listed below suggest 
types of information the planning committee might review* Many of the measures cited 
in the collection and services categories are discussed in OMPL, second edition. 
Libraries that lack information for these measures may gather that data now or in 
later objectives cycles. 
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Building: age; condition; code compliance: patterns of traffic for workflow and patrons; 
signage; parking; appearance; location; handicapped access; visibility; space needs for 
the collection, patrons, and staff; and adequacy of mechanical systems. 

Collection: breadth and depth of library holdings; formats available; use patterns; 
age of the collection; condition of materials; the rate of additions or withdrawals; 
adequacy of the catalog (either online or manual); and measure of collection use 
such as fill rates and turnover rate. 

, ServicC s: number and type of services; number of patrons using services; effectiveness 
and extensiveness of services and programs (see OMPL, second edition); studies of 
particular service areas, such as the reference department; and interlibrary loan 
statistics and patterrs. 

. Staff: educational levels; salary and benefit comparisons with similar libraries; 
length of service time mth the library; satisfaction with current library services or 
conditions; staff workloads; and opportunities for framing or continuing education. 

. Management efficiency and effectiveness of internal operational units (cataloging, 
acquisitions, administration, etc.); management skills; administrative procedures; 
relationships with other libraries and networking activities. 

• Budget allocation: adequacy of budget; comparisons with similar libraries; and 
analysis of resource allocations among budget categories (personnel, collections, and 
operations), among branches and the central library, or among services (Children's, 
Young Adult, or Adult Services; Technical Services; Outreach or Audiovisual 
Services, etc.). 

Step 3: Decide What Information Is Needed 

To decide what information is important for planning, the planning committee 
identifies a focus for ils information-gathering activities and agrees on the final list of 
information to be gathered. This manual presents two tools for use in this step: a list of 
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basic community and library characteristics (see "Information to Support Planning for 
Children's Services" following this article) and an information needs list (Figure 2). 

The amount of potential information about the library and its community is 
staggering. The planning committee needs to make sure it gets the right information 
—the information most imports and most useful for decision making. Too much 
information may overwhelm the committee, and too little information or the wrong 
information can mislead the committee or limit its vision. Identifying a focus for 
collecting the information helps the planning committee make its Looking Arouud 
activities as productive as possible. Libraries Looking Around at a basic or moderate 
level of effort can use Workform C, Options for Library Planning Information (see 
"Information to Support Planning for Children's Services" following this article), to 
focus their collection of information. The information needs list (Figure 2) is most useful 
at an extensive level of effort. 

Using Workform C 

Workform C presents a set of community and library characteristics useful for many 
libraries. Typically, the information shown is readily available from printed 
information sources, agencies and organizations, and existing library statistics. 
Planners can review these characteristics, weigh their relative value for the library's 
planning, and select the most useful items. 

Libraries planning for the first time or those Looking Around at a basic level of 
effort may use Workform C as a menu, selecting items of interest. Information not 
collected now can be gathered in later objectives cycles. Libraries Looking Around at a 
moderate level of effort collect most items on Workform C and make an extra effort to 
collect comparative information. 

Comparative information helps place current data in perspective. The 
"Comparative" column on Workform C could contain as appropriate: historic information 
(such as library service statistics from last year or 5 years ago); projected community 
data (such as population projections for 5 or 10 years into the future); or comparisons with 
other libraries (peer libraries within the state or nationally). Use whatever 
comparative information is meaningful and useful for your library's planning purposes. 
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Using the Information Needs List 

libraries Looking Around at an extensive level of effort can use an information 
needs 1* t to keep track of information of potential planning interest. Figure 8 shows 
part of such a list prepared by a library placing a high priority on circulation of 
popular materials and services to students. Each library's list varies, reflecting its 
unique needs. 

To develop an information needs list, the planning committee brainstorms and lists 
ideas for information that seem likely to be useful in planning. If possible, suggestions 
for the potential use of information items can be added to the list during the discussion, 
ideas for information items can come from the committee's review of Looking Around 
Outside and Looking Around Inside, Workform C, or a consideration of the following 
perspectives: 

• Planning committee perceptions 



• Library measures 

• Library roles 



PLANNING COMMITTEE PERCEPTIONS 

Just as librarians are aware of their library's current service profile, members of the 
planning committee each have a sense of the community's history, neighborhoods, and 
businesses. They understand its social and economic life, changes that are occurring, and 
the community's values and priorities. 

These perceptions can help generate ideas for needed information. In a community 
experiencing dramatic change (such as a town with boom/bust cycles), economic 
information may be vital for planning. A library in a community with a wide variety of 
educational and cultural resources may choose to look at its relationship with other 
providers of education and information in the community. 




LIBRARY ROLES 

During the next phase of the planning process (see Chapter 4), the library adopts 
roles. Most librarians already have a sense of the roles currently emphasized in their 
service programs. This sense can guide the gathering of information. For example, a 
library that places a high priority on services to elementary and secondary students is 
likely to be interested in the Formal Education Support Center role. This library might 
gather in-depth data about school enrollment trends, student patterns for using the 
library, school faculty awareness of library services, etc. Library planners may review 
the "Output Measures to Explore" and the "Critical Resources" described for each of 
the eight roles in Chapter 4 to help identify information useful for assessing the 
library's capacity to perform selected roles. 

LIBRARY MEASURES 

Measurement is the collection of data representing the state of the library, its 
services, and its users, v^itput measures provide objective data on the effectiveness of 
library services. Output measures are powerful tools for describing the library. They 
evaluate th: services the library delivers and can help assess current levels of 
performance, identify problem areas, and monitor progress toward objectives. 

Some libraries will choose to calculate basic performance measures derived from 
existing library statistics, such as circulation per capita, as part of Looking Around. 
Others may take time to complete more complex measures requiring surveying, such as 
fill rate measures. OMPL, second edition, contains piooedures for these and other 
measures that can provide information useful during Looking Around. 

When a first draft of the information needs list is completer, the planning 
committee assesses its information appetite. How much effort will it take to collect the 
information. How does that effort compare to the original level of effort set for Looking 
Around? What are the benefits of the information listed in relation to the cost of 
gathering it? To answer these questions, it may be useful to group similar information 
items together. This can help the planning committee identify any redundancy or gaps 
in the list. If the first draft of the information needs list is too lengthy, the planning 
committee should trim the list. 
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Step 4: Gather the Information 

Once the planning committee has decided what information it needs, the assigned 
library staff gather the information. This step includes organizing the information t > 
be gathered, tapping existing information sources, and completing other 
information-gathering activities as needed. 

Organize the Information to Be Gathered 

First, review and organize the informatics items the planning committee has 
selected. A useful way to group the items is by the potential source or method of 
collecting the information. For example, one group consists of the information to be 
found in current library records. Another might include information about the 
community available in census data or at a local or state agency. 

A third grouping includes data that can only be obtained through completing 
procedures suggested in OMPL, second edition. For libraries Looking Around at an 
extensive level of effort, a fourth group might include surveying or other activities to 
collect original data. 

Tap Existing Information Sources 

A wide variety of existing information sources is available to library planners. 
Always use the most current, authoritative source available. High-quality 
information contributes to effective planning. Out-of-date population data, inaccu. ,e 
financial information, or inconsistently compiled library service statistics are serious 
barriers to sound decision making. 

PRINTED INFORMATION SOURCES 

For city and county libraries, a variety of demographic information from the most 
recent census is readily available in the City and County Data Book. Other census 
publications, the statistical publications of state agencies, city or area directories, 
telephone books, atlases, almanacs, and newspapers are all possible sources. 
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GOVERNMENTAL AGENCIES AND OTHER ORGANIZATIONS 

Local and state government offices are often good sources of information. Check 
with the state library; planning offices, economic development bureaus, tourism 
agencies, local boards of education, rnd tax or revenue departments. They may have 
updates to census information, demographic, economic, and business information, or 
information derived from community needs assessment or other planning studies* 
Nongovernmental agencies can provide information too. Chambers of commerce and 
trade associations assemble information on the community and the surrounding area. 

In addition, planning documents developed by local government agencies, such as 
school districts or parks departments, often contain goals that affect library services. 
Statewide plans for library service developed by the state library agency or state 
professional association may also be useful. 

LIBRARY STATISTICS AND LIBRARY LITERATURE 

Current library statistics can provide a variety of information, and many library 
measures can be calculated using existing library records. Trends in these statistics and 
measures over the past few years can help the planning committee place in perspective 
and interpret current library performance. Check to see what statistic information 
about library services and trends is available on a regional, state, or national basis. 

A review of library literature may reveal a variety of useful reports. National 
studies of libraries, library users, and adult readers may be of interest to the planning 
committee. Or, the staff may be able to locate articles providing information about a 
topic or service area of concern to the planning committee. 

Complete Other Information-Gathering Activities as Needed 
Many libraries, particularly those Looking Around at a basic level of effort, can 
find adequate information in existing information sources. Libraries devoting a greater 
level of effort to Looking Around often engage in original data collection. Options 
include: 
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Library walk-arounds and community drive-around* Planners follow a route 
carefully planned to reveal the library or community in a comprehensive fashion. 
They look for answers to such questions as: What was surprising? Where was 
change evident? What types of people were encountered in the community, the 
library, or on the library staff? 

. Surveys: Library surveys obtain information directly from individuals such as 
library users, students, and staff. Surveys may be done in-person, by telephone, 
mail, or by a variety of drop-off and pick-up methods. Surveys are flexible, 
adaptable, and efficient, but require careful design, pretesting, administration, and 
interpretation . 

. Interviews: Interviews, like surveys, supplement statistical information by 
reflecting the views of individuals who might otherwise remain silent and by 
broadening citizen participation in library planning. Interviews may be targeted at 
groups or individuals. Effective interviewing requires care in selecting those 
interviewed, detenrdning interview questions, and organizing and interpreting the 
results* 

. Structured observation: Structured observation is a technique that studies specific 
activities, tasks, or events. Unlike surveying and interviewing, individuals 
performing the tasks are generally unaware that their activities are being 
analyzed. Like the techniques above, structured observation requires thorough 
preliminary planning and sound data collection instruments. 

While such activities are beyond the scope of this manual, additional information 
can be found in the citations at the end of this chapter. Assistance may also be 
available from your state library or regional library cooperative system. 



Step 5: Study the Information 

Now the most challenging part of Looking Around begins. Again, both staff and 
planning committee have an important role. The library staff organize and present the 
results of the infoimation-gathering activities to the planning committee. The 
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planning committee then translates this new knowledge into a form useful for decision 
making. 

Organizing the Information 

At this stage, the results, or findings, of the library's information-gathering 
activities are largely unorganized. The next task of the staff who gathered the 
information is to determine how best to organize the information for presentation to the 
planning committee. In general, group information items in a way that will simplify 
interpretation and use. Several schemes for organizing the information are possible. 

• Service orientation: Relate information items to selected library services. 
Information items that apply to more than one service may appear several times. 

• Roles orientation: Libraries that used roles (see Chapter 4) to focus information 
gathering may use those same roles to organize the information gathered. Again, 
information items relevant to more than one role may be restated with each role. 

• Topical outline: Use the general scheme presented in Workform C (see 'Information 
to Support Planning for Children's Services/' following this article) or, if major 
trends are evident, organize the information in terms of these changes in the 
library's community or services. < 

Try looking at the information from several different points of view. Group and 
regroup the information. Break down figures in alternative ways. For example, try 
looking at circulation per hour open or dollar spent on materials per day. It is not 
necessary to interpret the information at this stage, but the information should be 
presented to the planning committee in a clear, logical manner. This facilitates an 
orderly review of the information and helps the planning committee keep its attention 
on the most important findings. 

Interpreting the Information 

Now that the information has been gathered and organized, the planning 
committee must decide what this information means. It translates the findings into 
needs, opportunities, and library decision area*. This process is not a straightforward 
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matter. A description of "what is" does not lead « ctly to clear-cut agreement about 
"what should be." Interpretation is a complex task calling for reflection and judgment. 

The findings focus discussion, but the committee will need to think both 
analytically and creatively about the meaning and implications of the findings. 

Workform D, Translating the Findings of Looking Around (see Figure 4), can assist 
the committee in this process. The committee may spend one or more meetings discussing 
the findings and recording their conclusions. As the planning committee uses Workform 
D, they: 

• Ust in capsule form each of the major findings 

. Note briefly how this finding might impact library services 
—either in terms of the current situation or the potential for 
change 

. Given these impacts, determine if there are any opportunities or 
circumstances implied by the findings that the library should 
exploit 

• Consider how the library might respond to the findings. 

During the discussion, some general points should be kept in mind. First, both 
factual and subjective information are important. The impressions and opinions of 
individuals must be combined with "hard" data, such as population statistics or 
economic information. Subjective information places factual data in perspective, and 
factual data challenges or substantiates subjective impressions. 

Second, watch for surprises. Some information may confirm staff or committee 
impressions of "the way things are." Other information may, however, challenge 
long-held assumptions. Such areas should be reviewed carefully. They may indicate 
areas where change is especially needed. 

Third, try to place all information in perspective. Sometimes, to interpret statistics 
about the library's performance (such as circulation per capita or document delivery 
figures), it may be helpful to know what is possible. Such statistics may have more 
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meaning if they are compared to the library's past performance or to statistics from 
similar libraries. 

Fourth, libraries that have conducted surveys should remember that surveys are 
not ballots. The planning committee, director, and board are not honor bound to act 
directly on individual survey items. Often those responding are not aware of all the 
facts or constraints under which library services are provided. The opinions and 
perceptions of library users and the community at large are useful in planning, but not 
binding. 

Workform D provides a foundation for concluding this planning phase and for 
continuing the planning process. Chapter 4, Developing Roles and Mission, uses the 
findings as the planning committee considers which roles are mt Jt crucial for 
successfully meeting community needs for library service. As the library moves further 
ahead in the planning process, the ''Opportunities" column of Workform D may suggest 
areas for goals and objectives (Chapter 5), and the "Possible Library Responses" column 
may suggest specific activities for implementation (Chapter 6). 

Step 6: Report the Results 

Finally, the results of Looking Around are summarized. This summary serves as a 
permanent record for the current committee and for future committees. The length, 
complexity, and format of the report varies with the library's level of effort. Libraries 
reporting at a basic le /el of effort can use a "cieaned-up" version of Workform D to 
transmit the results of Looking Around to subsequent planning phases. For these 
libraries, Workform D will become a key source document for the final planning 
document (see Chapter 7). 

Libraries Looking Around at a moderate or extensive level of effort may produce a 
more lengthy report. Figure 5 suggests the basic content for such a report and can be 
modified to match your library's needs. Suggestions for adapting this outline by level of 
effort are provided in the level-of-effort box accompanying Step 1. General guidelines 
for the report are: 
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• Begin with a brief executive summary giving an overview of the 
report's contents 

. Keep the language and tone of the report simple, clear, and direct 

. When reporting statistical data, rely on tables, pie and bar charts, 
andXVvisufls (individually numbered and titled) rather than 
lengthy paragraphs that simply list numbers or percentages 

. Explain the implications and importance of each chart, table, or 
summary presented 

• Have someone knowledgeable about the library but not involved in 
Looking Around review the report to make certain it is readable and 
understandable 

The library has now examined its external environment and internal conditions. 
This Looking Around has resulted in a statement of key findings which have been 
discussed and interpreted by the planning committee. These findings, their potential 
impact, the opportunities noted, and possible library responses pro /idc input for the 
next planning phase, Developing Roles and Mission (Chapter 4 ), and for the planning 
phases to follow. 




Sources for Additional Information 



Busha, Charles H., and Stephen P. Harter. Research Methods in Librarianship. New 
York: Academic Press, 1980. 

[Chapter 3 provides an overview of the techniques and principles involved in 
survey design and administration. Chapter 6 reviews additional research methods 
including observation case studies, library user studies, library and community 
surveys, etc.) 

Dillman, Don. Mail and Telephone Surveys: The Total Design Method. New York: 
Wiley, 1978. 

[Although its cost data is outdated, this invaluable work provides step-by-step 
details for how to conduct successful mail and telephone surveys. It contains 
numerous checklists and examples helpful to the novice.] 

Epstein, Irwin, and Tony Tripodi. Research Techniques for Program Planning, 
Monitoring, and Evaluation. New York: Columbia University Press, 1977. 
[Libraries interested in structured observation may wish to review Chapter 5, Using 
Observational Techniques for Planning Staff Training Programs. This chapter 
discusses the principles of structured observation, applies those principles, and 
cites a specific application as an example.] 

Gallup Organization, inc. Book Reading and Library Usage: A Study of Habits and 
Perceptions. Princeton, N.J.: The Gallup Organization, Inc., 1978. 
[This publication is an example of studies in the library literature that explore 
patterns in reading and library usage.] 

Hernon, Peter, and Charles R. McQure, editors. Microcomputers for Library Decision 
Making. Norwood, NJ.: Ablex Press, 1986. 

[Useful chapters for libraries intending to use microcomputers during Looking 
Around, include Chapter 6, Designing a Spreadsheet Decision Support System in a 
Library, and Chapter 12, Microcomputers for In-house Data Collection and 
Research.] 

McClure, Charles R., Douglas L. Zweizig, Nancy a. Van House, and Mary Jo Lynch. 
Output Measures: Myths, Realities, and Prospects. Public Libraries (Summer 1986): 
49—52. 

[The article gives an overview of the issues involved in determining what quality 
of information is appropriate for use in library management, and gives suggestions 
for interpreting measures data.] 
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Palmour, Vernon E., Marda C. Bellassai, and Nancy V. De Wath. A Planning Process 
for Public Libraries. Chicago: American Library Association, 1980. 
[Part III, Collecting and Using Data, discusses surveying as a method for gatiutfrg 
planning informarion. Examples of surveys for staff, citizens, students, and 'aoraiy 
users are presented along with suggestions for tabulating and analyzing the results.1 

Swisher, Robert, and Charles R. McClure. Research for Decision Making. Chicago: 
American Library Association, 1984. 

libraries interested in gathering original data through ^f^^^^f 
find this publication useful. Chapters 3-5 discuss research design, data gathering, 

and sampling.] 

U.S. Bureau of the Census. County and City Data Book, 1983. Washington, D.C.: 
Government Printing Office, 19M. , , . _ 

TOs pubUcation combines a wide variety of social and economic data from Census 
Bureau publications. Most information is available by state, county, city, and 
standard metropolitan statistical areas.l 

Van House Nancy A., Mary Jo Lynch, Charles R. McClure, Douglas ZweWg, and 
£5 JoFodger.O«5«t Measures for Public libraries. Second edition. Chicago: 
American Library Association, 1987. 

[This publication provides a useful overview of the basic principles for surveymg 
and sampling, such as usefulness, validity, reliability, precision, "^ability, 
etc. It gives basic procedures for deriving the output measures referenced in this 
chapter and suggests approaches for reporting, mterpreting, and using measurement 
data.] 
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FIGURE 1 

General Do's and Don'ts for Looking Around 



DO: Consider both factual data and subjective impressions about the library, its 
services, and its community. 

DO: Take advantage of information already collected by others both within and 
outside the library. 

DO: Know in advance what use will be made of each piece of information 
collected. 

DO: Allow ample time to think about and interpret the information gathered. 

DO: Set a clear time schedule for Looking Around activities. 

DO: Refer to Output Measures for Public Libraries, second edition, for measures and 
data collection methods suitable to the library. 

DON'T: Allow Looking Around to become an end unto itself-— all steps in the process 
are important. 

DON'T: Collect more information than the planning committee can use. 

DON'T: Conduct surveys unless the library can commit an extensive level of effort to 
Looking Around and has (or acquires) the necessary expertise. 
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FIGURE 2 
Sample Information Needs List 




City and County Data Book 
or State Planning Office 



Community Population: 
% under 5 years of age 
% 5-17 years of age 
% over 65 years of age 

Total Registered Borrowers Library records 

Any changes? 

% of community aware of Survey 
various library services: 

Basic circulation services 

Children's story hours 

A/V services 

Kiosk displays in mall 

Reference services 

Interlibrary loan services 

Meeting rooms 

Turnover rate: our library Library records 
Turnover rate: other lib's State Library 



Check projected student population 
Check current student population 
Has senior population grown in the 
last 5 years? 

Get current + 5 years ago 



Is the library well known? 
Do we need to publicize some 
services better? 



Library Circulation 
Adult 
Juvenile 

% of students (K-12) who 
have visited the library: 
within the last month 
within the last year 



Library records 



Survey (Get data at 
the building level) 



How active is the collection? 
Comparisons with similar size 
libraries in state 

Who accounts for what part of our 
circulation: 



Which schools account for most of 
our student use? How could we 
target a library /faculty 
relations program? 



...and so forth 

Subject "and Author RH Rate* "oMPL, 2nd ed./Survey How well are we doing? 
Ratio of additions to total Library records 



library holdings 



Help in diagnosing the currency of 
the collection 



NOTE: Prepared from the point of view of a library emphasizing circulation of popular materials 
and services to K-12 students. 
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Figure 3 

Level of Effort for Looking Arounu 



Basic 

The library director assigns responsibility for collecting and organizing Looking 
Around information to a staff member. In very small libraries, the director may handle 
this task. The planning committee uses Workform C as a menu to select the information 
useful for its planning purposes. These items are gathered from existing information 
sources and organized by the staff. The planning committee discusses these findings and 
uses Workform D to identify the potential impact of the findings on library services 
and to determine possible library responses. This workform serves as the report of the 
Looking Around planning phase. 

Moderate 

The library director assigns responsibility for coordinating the collection and 
organization of Looking Around information to a staff member. Other library staff 
assist. The planning committee selects most items listed on Workform C and obtains 
trend and comparative information for many of the items. These items are gathered 
from existing information resources and organized by the staff. The planning committee 
discusses the findings and uses Workform D to identify the potential impact of the 
findings on library services and to determine possible library responses. The report of 
Looking Around activities and findings is brief, but follows the basic outline shown in 
Figure 5. 

Extensive 

The library director assigns responsibility for collecting and organizing Looking 
Around information to a staff committee. In addition to the items listed on Workform C, 
the planning committee develops an information needs list containing a broai range of 
additional inforr ion. Some of this information requires library staff to collect 
original data through measurement activities (such as those described in OMPL, second 
edition), surveying, interviewing, or other data collection methods. The staff use 
research skills and computer support to analyze the data, determine findings, and 
produce summary graphics. The planning committee discusses these findings at length, 
using vVorkform D to summarize the results of its discussion. The staff prepare a 
comprehensive report of Looking Around activities and findings based on the outline 
shown in Figure 5. In addition to its uses in the planning process, this report may be 
widely disseminated to the board, staff, local government, and the community. 
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Figure 4 

Workf orm D: Translating the Findings of Looking Around 



Major Finding 



Impact on Library Roles! Opportunities? 
and Services 



Possible Library 
Responses? 



1. 



2. 



3. 



etc. 
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Figure 5 

Suggested Outline for Reporting the Results of the Looking Around Process 



Executive Summary 

• Summarize in one or two pages the basic findings, their meaning, use in the planning 
process, and the overall value of the process. 

Introduction 

• State the general focus for the Looking Around process: what did the library want 
to find out? 

• Describe how the Looking Around process was managed and over what time period. 
Approach 

• What Looking Around activities were accomplished? 

• Briefly report what procedures and existing sources of information were used, what 
agencies were contacted, and what other methods were used to gather information. 

• How were data collection instruments, if any, developed? 

• Explain how the information was analyzed. 

Findings 

• State the findings, and present summary tables. Organize the findings by broad 
topics (such as the roles described in Chapter 4), major library service or program 
areas, or by the results from particular Looking Around activities. 

• Give an overview of what the findings mean, how they impact library services and 
operations, and discuss the implications of the findings for library planning. 

Appendixes 

• Copies of any data collection instruments used. 

• Support documents for Looking Around such as time schedules, explanation of 
statistical procedures used, etc. 

• Any other materials, such as maps, that might be useful in explaining the Looking 
Around process. 



NOTE: To be used for moderate or extensive level of effort only. 
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Information to Support Planning for Children's Services 

(Adapted from Charles R. McClure et al. Planning & Role Setting for Publk ^«*« 
AMaZl of Options and Procures. Chicago: ALA, 1987 by Douglas L. Zwdzig, 1989.) 



Date 



% 



Library^ _ — ^ Give date i^nnation was obtained.) 

tSSZ^S** els/akl in ta.erprs.ing .he results of children's «mces pregrem 
evaluation measures. 

Community Characteristics* 

Information on individuals 

Population of legal service area — 

# in legal service area under 5years of age 

% in legal service area under 5 years of age 

# in legal service area 5 to 9 years of age — 

% in legal service area 5 to 9 years of age 

# in legal service area 10 to 13 years of age 

% in legal service area 10 to 13 years of age — 

TOTAL: 

# in legal service area 0 to 13 years of age 

% in legal service area 0 to 13 years of age 

# Data on community characteristics may be found in the US. census reports, city or county 
SSg d^Zents, school district offices, regional cooperative government agenries 
SSmd i Splaiming, Chamber of Commerce, social service agencies. Your state library 
agency may be a particularly useful source. 

For future reference, indicate any additional sources found helpful: 

Indicate those sources found best for this information in your community: (Remember to date 
this sheet.) 



% 



% 



% 
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Information to Support Planning for Children's Services 

Librar y Date / / 

(Give date information was obtained.) 

Community Characteristics (continued) 
Educational level of the community 

(% of adults (25 years +) completing high school) % 

Other (Cite other statistics or characteristics, such as language or ethnic minorities.) 



Information on families and households 

Total number of households in legal service area 
Average number of persons per household 



Percentage under 18 living with two parents % 

Median family income $ 



Percentage of families below poverty line % 

Nurseries and day care centers (If too many to list here, cite a directory.) 

Name Enrollment 



# of licensed family day care homes 
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Information to Support Planning for Children's Services 

Lib Date / / 

iiiurary^ _ . ^ Jve date ^nuation was obtained.) 

Community Characteristics (continued) 

Agencies thatserve children (YMCA, Scouts, community centers, social agencies, etc) (If 

too many to list here, cite a directory.) 



Schools— public and private (If too many to list here, cite a directory.) 



Name Public or Private Grades Enrollment 

[check one] covered 
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Information to Support Planning for Children's Services 



Library_ 



Date 



/ 



/ 



{Give date information was obtained.) 



Library Characteristics 

Total budget for children's services 



What are the key categories of expenditures (e.g., personnel, materials, equipment, 
supplies, contracted services) for children's services? 



Expenditure category 



Total operating budget of the library 

Percent of total operating budget spent 
for children's services 

Per capita support for children's services 

Per capita support for general library services 



Amount budgeted 

$ 

$ 

$ 

$ 

$ 

$ 

$ 

$ 

$ 



$ 



% 



./child 
/resident 
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Information tc Support Planning for Children's Services 



Library. 



Data 



(Give date information was obtained.) 



Library Characteristics (continued) 

Sources and amounts of funding for children's services (LSCA and other grants, private 
hinders, library Friends, etc.) 

Amount provided 



Source of funding 



Staff hours and level of staff assigned to program 
Level of staff 



$. 



Hours per week 
dedicated to 
children's services 



Space for children's services 

Square feet of space in public service area 
Square feet of space for meeting rooms 
Square feet of space assigned to staff offices 

Totalsquare feet of space assigned to children's services 




Information to Support Planning for Children's Services 
Library Date / /. 



(Give date information was obtained.) 
Library Characteristics (continued) 

Collections 

Size of collection by type of material: 

Material type Number of items 



Currency of the collection: 

% of non-fiction materials published in last three years 
(Could adapt the Title Availability Sample method from 
Ellen Altman et al, A Data Gathering and Instructional Manual 
fot Performance Measures in Public Libraries, 
Chicago: Oberon Press, 1976.) 



% of collection added in last three years 



% of collection deleted in last three years 
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Information to Support Planning for Children's Services 

Date / I 

" Drar y. — (Give date information was obtained.) 

Library Characteristics (continued) 

Equipment available or used for the children's services program 

lc ... M v«,«» a «. Number of items 

Type of equipment for use by staff . 



Typeofequipmentforusebythepublic: Numberofitems 
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II. Systematic Observation 



Introduction 

To observe systematically means that one knows 

a) precisely what is to be observed; 

b) how it will be observed; 

c) how observations will be recorded; 

d) how they will then be analyzed; and, finally, 

e) how sense will be made of the analysis. 

Observations can be made through watching, administration of questionnaires and 
interviews or through diary-keeping among other methods. This chapter of the 
Sourcebook includes information on interviewing, conducting surveys, sampling, and 
questionnaire construction, including examples of questionnaires. In addition, this 
chapter includes material on an environment rating scale developed by Holly Yfillett, 
specifically for observing the environments) from which children's services are 
provided. 

The first item in the chapter is based upon an Institute presentation made by Jane 
Robbins. It covers the general observational technique of participant observation. The 
secon J item is adapted from the transcript of a presentation made by Lynn McDonald. 
It contains a large number of techniques that have proved useful in interviewing 
children. This information has already been shared by Institute attendees in a number 
of local library publications. It is followed by an explication of a specialized 
interviewing technique valuable for use with children called backward chaining; this 
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presentation was made by Eliza Dresang. The next section includes mere material from 
Jane Robbins' presentation, this portion on sampling and a reprint of a classic on 
sampling from Morris Slonim's Sampling in a Nutshell 

A number of items on questionnaires make up the next section including some 
additional material from Jane Robbins' presentation, an article by Barbara Moran on 
constructing questionnaires, and some examples of questionnaires designed to deal with 
children's services. The final section is the Willett environment rating scale which 
has already proven to be of high interest. 
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Systematic Observation 



Jane Robbins 

Systematic observation requires first that you clearly and precisely know what it is 
you are attempting to observe as opposed to just being in the children's room hoping that 
something will hit you between the eyes. This is not to discount the value of the 
interocular traumatic test; in other words, if something does hit you between the eyes , 
then it probably is something to which you ought to be paying some attention. 
However there is also the kind of observation where you are looking for the activity 
that is happening constantly, but that you no longer see. Observing the ordinary is also 
a very important part of observation. For example, you may d^dde that it is important 
to determine what children really do when they are in the children's room. You may 
think you know what children do, but unless you take some time to observe 
systematically, you probably cannot explain to others with conviction. 

Systematic observation is done differently by different people. For instance, some 
people find dichotomies as opposed to continua the best way to understand concepts or 
events. People who find dichotomies most useful would describe children in the library 
as either boisterous or quiet even though they know that children, most of the time are 
somewhere on a continuum between boisterous and quiet. In terms of a systematic 
observation continuum, at one end are positivistic observers. These are researchers who 
are seeking to understand through facts in order to attribute cause. They want to know 
precisely what makes a child come to the library. What is the cause or are the 
integrated causes that make a child come to the library? They want to collect data in 
order to be able to understand and explain, even to predict to others which children arc 
most likely to come to the library. Most positivistic researchers tend to want 
quantitative answers. Thaf s not to say that other researchers are not quantitative, but 
only that positivists tend to prefer to look at things quantitatively. 

At the other end of the continuum are researchers who are more phenomenological. 
What phenomenological observers try to do is to seek to understand how children 
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experience things. To a phenomenologist, reality is not so much facts but rather what 
people imagine things are; reality is what is believed as opposed to what is. To the 
phenomenologist what is is irrelevant; if s what people believe it to be that is 
relevant. Observers who are phenomenological are more likely to be qualitative in 
terms of the material they use for their research. 

Questionnaire and interview data can be used either quantitatively or 
qualitatively. There is nothing inherent in a data collection method that requires the 
data to be used in just one manner; therefore, in many respects, referring to qualitative 
and quantitative researchers is not particularly useful. To denigrate qualitative 
research because it may not be based on quantitative data obfuscates the point of doing 
research; that is, to systematically question so that others can understand how it is 
that you arrived at your conclusions. When you talk to those who are unfamiliar with 
children's services and you say: "The children who come to this branch prefer hearing 
stories and the children who go to this other branch prefer craft activities/' some will 
probably say: "Well yes, thaf s probably right. You have a lot of experience." Yet 
others would ask you, "How do you know?" If one of the ways you know is that you 
spent a certain amount of time doing direct, systematic observation then your own 
confidence in your reply may be increased— and certainly the extent to which your 
response is held true by others will be increased. 

Observation is done so that you can determine what is. Observation for evaluation 
is to shed evidence, not to prove but to improve; to describe a phenomena of interest 
accurately, not necessarily emphatically. 

One of the possible pitfalls in approaching observation systematically is a 
tendency to get lost in the process of observation. It is important to remember why we 
are observing. It is so easy to forget the end when we focus on the means. My favorite 
management text is Robert Townsend's Up the Organization. One of the techniques he 
used when he was president of the Avis Corporation was to keep a sign by his phone 
that read: "We rent cars." He did this so he would not forget what it is that the Avis 
Corporation does. In the systematic observation of children you must not let the 
recording on a form become the end instead of the means. 
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What precisely does systematic mean? It means that you know what you will be 
observing; you know how and when it will be recorded and who will be doing the 
observation. You have determined procedures to insure accuracy in recording. Related 
to accuracy, for instance, you wouldn't want to assign someone to observe children for a 
four hour period, nor would you want the observer to interfere with the natural 
behavior of the children. Also, you undoubtedly would not want to try to observe 
eveiything thaf s going on in the children's room during a period of time but rather pick 
out one or more activities or groups to observe. For example, you might decide to observe 
either children or caregivers, or your observation might focus on what proportion of the 
caregivers who come in with their children are actually involved with what their 
children are doing. Is there a time of day when caregiver interaction is more likely to 
happen? The results of this type of observation could have some impact on staffing. If 
what you are attempting to observe is the interaction between children and caregivers 
then systematic observation requires that that is what you must focus your observation 
on. 

Of course, there are many other kinds of activities you might be looking at What 
are the characteristics of the people or events you are trying to observe? Perhaps one of 
the events t it you observe is that children are often having a hard time reaching 
material. Wlu,t you are really observing is the use of the physical facilities (too high 
shelves), as opposed to the behavior of individuals. 

Very often what you think you are seeing, what is apparent, is not real. Sorting out 
the apparent from the real is one of the things that systematic observation helps you to 
do. As an example, how can you tell the difference between a young adult who really 
only has jeans torn at the knee to wear and the young adult who wears those jeans 
because that's the current "right way" to dress. That is just an example of: "if you don't 
know what the context is, sometimes what you see or what appears is not what is." 

Observational accuracy requires practice; simply listening, which is a skill in and 
of itself, requires practice. It is possible to learn to listen more accurately. Have you 
ever been with a child who was talking with you while you were going through some 
papers or a book on your desk and you told the child it was okay to keep talking because 
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you were listening. And you really were listening until your shopping list (the one you 
lost last week) falls out of the book or shows up in the papers, and then all you can 
think of is: 'There's that list/' The result of course, is that you didn't hear the last 
sentence the child said. You can do two things at once, so long as nothing important or 
unusual happens in one area. Listening well is a critical observation skill and one that 
can be improved. 

If you are not a skilled listener, you can tape record interviews; then you don't have 
to worry, because you can go back and really listen to the tape recording. The problem is 
that the body language isn't there with the tape recording, and so often it is body 
language that makes the difference in terms of observing what is really taking place. 
You can, if what is being observed is very important, have more than one observer at a 
time. Then both observers can check to see whether, using the same method of recording 
observations, they have tended to see the same things. 

Another way to increase observational accuracy is to remind yourself of what you're 
looking for. Just before you start and again about halfway through your period of 
observation, it is important to state, for example: "What I'm doing here is: watching 
children interact with their parents, not interact with other kids, it's only the 
parental interaction that I'm looking at/' Another way to check accuracy is, after 
you've got some observation material, to tell the children what you've seen, and ask 
them whether thaf s what they are doing. They are capable of feeding back, and 
saying, "What? You thought I was talking to my mother? I hardly ever talk to my 
mother, but she talks to me/' 

There are many good ways one can observe things that are going on in the children's 
room. As far as I know though there has been very little direct observational data 
collection in libraries, and virtually none has been done in children's rooms. 
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Interviewing Young Children 



Lynn McDonald and Holly Wiilctt 

Lynn McDonald has been doing a workshop on interviewing children which has 
been offered through the Universi y of Wisconsin-Madison Extension for the last two 
years. The people who have attended this workshop have been social services 
workers, public health nurses, lawyers, police officers, and teachers but no librarians. 
This is the first time the concerns of librarians who include children in the focus of their 
work have been addressed. 

The presentation is structured around the following six topics: 1) The first question 
will be, why is it important to think about interviewing skills with children if you're a 
librarian? 2) Then we'll ask, why is it hard to interview children? McDonald will 
suggest that children come from a culture that is different from our own, and, because of 
that, we need to know about their culture and use techniques that overcome the cultural 
diversity that children present to us. 3) Next, we will contrast interviewing techniques 
used with children versus those used with adults. 4) We will then present cognitive 
and verbal characteristics of children under eight years old which are applicable to 
your own interviews with children. 5) Next, we will describe how to interview 
children. 6) Finally, we'll confront difficulties by sharing ideas of what to do when 
they don't talk, and what to do when they talk too much! 

Many of our examples suggest interviewing children about books you have read to 
them, but this is not the only service about which children under the age of eight could 
be interviewed. We recommend interviewing young children for the purposes of 
evaluation only when the results are likely to produce information that can be used to 
improve services to them. Children's librarians have a strong professional ethic 
against intruding on children's privacy in selecting and responding to library services 
and materials; so interview questions need to be framed with this in mind. 



UNIVERSITY OF WISCONSIN— MADISON M»cDaruld k Wflktt 

115 

llii 



The Importance of Interviewing Children 

The first question is why should you develop interviewing skills with children. 
Part of the answer to this lies in the lack of services available to children. Research 
generally shows that although only 22% of all adults use the public library, 30 to 40% 
of all children use the library. Almost twice the proportion of children use the library 
than adults. Yet when you look at the funding per capita in library budgets, the 
funding for adult services is twice that of the funding for children's services. So, 
children are high users of library sen ices yet under-funded in terms of program. This 
pattern is seen throughout all services— not just libraries. There is often more money 
available to adults than kids, even when kids are higher users of certain services. In 
mental health and social services this is seen all the time. Is this right? No! What 
can we do about it? Advocate for children's rights! Anything you can do to support 
increased funding for children's services is important to pursue, and information 
gathered through interviewing children can provide some needed support by furnishing 
evidence of the effectiveness of services to children. 

Why It Is Hard to Interview Children 

Children's needs are underrepresented in the adult-dominated world. They have 
few advocates, and, as a result, they do not get the services that they need. 

One reason why children may not receive the services they need is that we as 
adults have difficulty relating to small children. And if we cannot understand a person, 
it is hard to advocate for their needs. For example, it is hard to interview kids. It's 
hard to deal with them. They interrupt us. They're messy. Have you noticed that 
children's services areas are messier after the end of the day? Children are slower. 
They don't understand us very well. They wiggle. They don't use words well. They 
distract us. If s hard for us to deal with children. Those of us who work with children 
or have children at home or nieces and nephews nearby or neighbors have a chance to 
become more familiar with children's ways, but we may still not fully understand 
children's culture. 

There is a story about a woman who was laying a concrete sidewalk. She carefully 
put out all the materials, smoothed it down, and got it just perfect. It was a sunny day 
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and she sat in her lawnchair and was calmly reading a book and having a glass of 
lemonade. All of a sudden she saw four or five children running towards her concrete, 
and they went smashing through it. She yelled and screamed at them, "Get out of 
there!" because she was very upset. A neighbor came by and said sarcastically, "I 
thought you liked children," and the woman replied, "I do, in the abstract, but not in 
my concrete!" Thaf s the problem with children— it's fun to talk about them and if s 
harder to be with them for long periods of time. 

One of the main reasons it is hard for adults to relate to children is because they are 
really culturally different. For heuristic purposes McDonald will describe seven 
concepts she uses to distinguish among cultures. Applying these concepts suggests the 
conclusion that children are culturally different from adults. 

The first concept is size and shape. Children are littler than adults are and have 
different physical shapes from adults. Their heads are too big, their necks are too 
short, they have no waists, no breasts, no body hair, no muscles, and their proportions 
are off. Most of all however, they are very small and they look different from us. 
Thafs number one. 

The second concept is language: they speak a language different from ours. This is 
how they speak a different language: We think preschoolers are about 80% non-verbal 
and 20% verbal, in terms of how they communicate. Most of their communication is 
non-verbal. With adults, we postulate the opposite: 80% of our communication is 
verbal and 20% non-verbal. Adults check the veracity of speech by watching for 
non-verbal cues, but our primary thrust is to trust the speaking. If there are 
incongruities between verbal and nonverbal you send the speaker to a therapist! With 
children it is just the opposite. They are just learning their expressive language, and 
their communication is mostly non-verbal. Knowing this is very important when 
interviewing small children. They like to touch things and have them be real in their 
hands. Their vocabulary is very small. When they're under five their language 
structure is still developing. By five we think their understanding of the structure of 
language has developed, and after that increasing their vocabulary is the main area of 
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language growth. There are lots of characteristics that would suggest they really 
speak a different language from our own. Knowing that we ivill be better interviewers. 

The third way that children's culture differs from adult culture is in what they do 
with their time. We talk and work. They play and go to school. They do different 
activities during the day. 

The fourth cultural concept is where they spend their day. Children have different 
physical facilities. There are different physical structures for them, and they are very 
segregated. One of the nice things about children's services in libraries from an 
outsider's perspective, is that at least they're housed in the same building. They are 
put over there, on the side, in the low wing without any funding, maybe, but they are in 
the same building. Most of the other things offered for kids are totally segregated. 
The only other place where adults and children exist in the same setting is in the 
family home. We have different physical structures for children, for what they may 
inhabit, and there are rules about whether or not they can move into the other ones. 
There are symbols which say whether the geographic structure is welcoming or not to 
the child. 

The fifth way in which kid's culture is different from adult culture is that they 
have different values from us. For example, they are not as neat as we are. They're 
slow. They believe in the here and now. They don't clean up after themselves. They 
seem egocentric. They're much more indirect than we are, except when they're 
evaluating us, and then they're very direct: "I don't like you!" They're not very polite. 
So, they have lots of different values from us. 

A sixth variable suggesting cultural difference between adults and children which 
may be unique is that as adults we think we know what children's culture is like, 
whereas children think most adults don't know children's culture. Adults are like 
immigrants. If you had been raised in Germany and then came to the United States for 
several years and subsequently went back to Germany, you'd discover that Germany 
wasn't as you remembered it. Because adults were all members of that child culture, we 
think we can remember, and so, we have some confidence about our ability to understand 
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children's culture. But as adults we cut off our memories and make them inaccessible. 
Children will tell you very openly if you ask »hem that really you have no business 
saying that you know their culture. It is a funny kind of cultural difference because we 
claim familiarity with the culture, even though we've lost our ability to function in it, 
usually. 

The last issue about children's culture that I wish to point out is that in this 
country, at least, there is a somewhat covert value message: children's culture is a 
deficit culture. Children are not just different, they are deficient. They haven't learned 
our ways, so we say children are "not polite" rather than saying they are 
"straightforward" or "not hampered by hypocrisy." Rather than saying that children 
haven't become caught up in verbal diarrhea, as adults can be trapped into, we say 
they're deficient because they don't know language yet. The negative value message 
imposed on children puts them into a powerless position: they are told that they're not 
enough, they're not good, they don't know anything, they don't know how to read, they 
don't know how to talk right, they don't have much 'nformation, they don't this, or 
they don't that. In contrast, the cultural diversity message could be one of equality and 
respect. For example, there is only one country in the world 'Jut has a law that you 
cannot hit a child, and that is Sweden. In every other country it is legal to hit 
children. The value-laden deficit model discussed here could be seen to actually 
generate violence. People will justify hitting children by saying, "I've got to get that 
kid socialized; there's something wrong with him." 

We want to put the following remarks in the context of these seven cultural concepts 
because to interview children is to address the issues we have raised. 

Interviewing Children Vs. Interviewing Adults 

What do children like in terms of interviews? McDonald informally asked 
children what they like in terms of interview behaviors. They say they like to be 
respected. They say they don't like grownups to be fake. They say they don't like 
grownups to be in a hurry all the time. Children want time, they want to be respected, 
and they don't like fake interpersonal behavior. These are the cornerstones of effective 
interviewing with adults as well. The variables which best predict successful 
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interviews with adults are: warmtn, genuineness, and empathy (Truax and Carkhuff, 
1967). Willett's experience interviewing librarians suggests that an open attitude and 
a willingness to accept answers as given elicit more information from interviewees. 

Cognitive and Verbal Characteristics of Children Under Eight 

Here are a few facts about preschoolers; knowing these should enhance your 
effectiveness as interviewers of young children. The first is, that until they're six to 
eight there is a huge imbalance of receptive and expressive language. Because their 
language is developing all the time, they can understand twice as much language as 
they can express. For example, if you read them a book they'll understand the words, 
but if you ask them to tell you they can't. It doesn't mean they didn't understand it. 
They just don't have the expressive capacity. This is part of developing language; 
children can understand a lot more than they can express. Thaf s so impot.. M to 
remember. Because they can't articulate it does not mean that they didn't understand. 
For example, using pronouns is very confusing to a preschooler. If you say, "What did he 
do?" or "she" or "it," they'll be confused. Be specific. "What did the bear do?" or 
"What did the big man do?" Don't use pronouns. Use simple words. Thaf s obvious, 
since you know that their vocabulary is smal 1 

Avoid double negatives; they're very confusing to the preschooler. Avoid "if" and 
"then" complex sentences. Preschoolers have concrete, literal thinking. They think ir. 
the here and now, and they're concrete thinkers. Preschoolers will answer yes to 
anything to please adults. Because they are so very interested in pleasing us, the 
standard answer to "Did you understand this movie?" would be "Yes." Big smile. But 
that is not how you are going to get useful information. They do not logically organize 
their thinking or their speech. Young children tend to ramble. If you ask a 
five-year-old child to tell you a story, she will take the full time to tell it that she did 
to hear it. Only ask if you have the time to listen. Don't ever take a five-year-old to a 
movie and then ask him to tell you the story because it will take two hours to hear it, 
and he'll be very upset if you interrupt him. 

Young children cannot keep two concepts in their minds at the same time. Their 
ideas about cause and effect are very primitive. In fact, until they're five and a half or 
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six their causal sequences will seem confused. Piaget describes the egocentric thinking 
of the two- to ave-year-old (Piaget, 1962). What that means is that the child thinks 
every causal chain includes himself. For example, if you turn the switch and the light 
goes on, the child thinks she had something to do with it. Or, if you walk around with 
a four-year-old and she looks up at the sun she'll say, "The sun is following me," or 
The sun rose because of me." There is the story of a ave-year-old boy in the New York 
City blackout who was hitting a little toy hammer against a telephone pole just when 
all the lights went out in the city. He ran home crying and wailing and weeping, 
"Mommy, I'm sorry, I didn't mean to." He just assumed that he was right in the middle 
of the causal chain. That's something to remember about preschoolers; if s just a 
mistake about how their brains work (using the deficit model). They include 
themselves in every causal chain. That's just the way they are, you can't talk them out 
of it. If they go through good experiences i.\ their lives, they think that they've caused 
them. If horrible things happen, they think they've caused them also. 

The young child learns only from direct personal experience. The child's 
understanding of space and time are non-linear and non-logical. If you want to talk to a 
child, don't talk to them about January, or September, talk about when school started, 
or when Grandma visited, or when their birthday was. Why? Because they're 
egocentric. (You may know some adults who are like that.) 

How to Interview Young Children 

How can you go about using these characte. sties of preschoolers in an interview? 
The first trick about how to talk to little children is, don't start by talking. There are 
several things you need to do before you can talk. First of all, you need to think about 
the environment that you have for them. Is it a child-welcoming environment? If you 
work in children's services, then it is; children's library areas are wonderful. The rest 
of the library building may seem huge to a child and may feel imposing and not 
accessible. However, children's services areas are usually very welcoming. With 
children, one or two token child-culture objects can transform the setting from alien to 
friendly. Any person from an alien culture responds similarly. If he enters a setting 
from a different culture, he looks for a person or object he can relate to to increase 
affiliation. Ifs the natural reaction especially when there is a discrepancy of power. 
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So if all the people in the room have more power, and one person wi v less power comes 
in, hell look to see if there's anyone else in that category of less power. As soon as 
there is, he feels better. 

You've seen that with children. If they walk into the library and see another 
child they feel better. It means if s safe. If there is no other child there and then they 
see something that looks like children's culture, they Ml feel better. Without seeing a 
welcome sign, children may assume non-welcome, and they may retreat and become 
difficult to interview. You then have to overcome the environment somehow yourself. 
McDonald always takes Httle child's culture objects such as small toys in her purse 
when she goes to do an interview, just in case the alien environment does not provide 
symbols for overcoming the negative cultural bias. 

So, the first thing is to think about is the environment On a more personal level, 
you can change how you are dressed and how you present yourself for working with 
children. Kids like color. Kids like things on you that are objects of interest One 
reason for that is that kids like to be indirect. They would much coking at your 

pin as they start approaching you than looking in your eyes. One ot u\e women who 
taught McDonald about how to interview kids said, "Watch what you smell like/' 
because lots of perfume turns kids off. Another expert said, "Be sure you're at their eye 
level." Children spend hours craning their necks because adults are always "up there." 
So, either sit down, or lift them up. Rather than having a chair that is the same 
height as the child's, yours should be lower, and theirs should be higher. 

You want eyes to be at the same level. So if you have a little platform, you could 
put the kids on the platform, and you sit below them* 

You want your nonverbal messages to be welcoming. Your body language is going to 
mean lots to them. Be patient, look comfortable, be spontaneous. It's better to 
underreact than overreact. Be supportive of them Children like it when adults look 
comfortable. That doesn't mean you have to wear jeans. 
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Having an interview with a child is what we adults want to have happen. It is our 
agenda; it is not the child's agenda. We start with something that we want to control 
for our purposes, and it is not their agenda. To be effective, alter the power structure of 
the encounter by saying, "Half of that time is for my agenda, and half of it is for the 
child's agenda." If you have an hour's worth of questions, then put in an hour for the 
child. If you have half an hour's worth of questions, put in half an hour for the child. 

It is very hard to convince busy adults to do this until they try to conduct their 
agenda without first responding to the needs of the children. Then they discover that 
it is useful in terms of efficiency to put in the half hour for the children first. It simply 
does not work otherwise. Children will stalemate you. They will sit silently and be 
quiet, and you will get nothing. The only way children have power is by doing nothing. 
They'can hold their mouths closed, and then they have power. If your goal is to make 
them talk, and their goal is to have some power, then you have to do a trade-off. You 
can give them some power at first, and then they will talk with you. 

What do you do to respond to the child's agenda? For the time that is the child's, 
you have to have some do's and don'ts. For the time that is theirs, 1) you cannot ask 
questions; that would mean you were in charge of the topic; 2) you cannot teach: "What 
color is this? What do you think this is made of?" You would be in charge then; and 3) 
you don't praise them. Praising is a hierarchical message as well: "I think you did a 
good job." You want to build an equal relationship, rather than a hierarchical one. An 
equal power message is an affiliative one: We are partners in the library, we are 
partners together. When McDonald trains people how to not do those things she uses a 
little microphone in the ear so she can give them immediate feedback, or she coaches 
them from behind. As you practice interviewing, use a third person to watch and correct 
you if you slip into teaching, directing, questioning, or praising. Have the third person 
say, "Wait, you can do that later, when if s your agenda." If s not that those things 
aren't fine to do, but rather that you have more impact it you let the child have her 
time first. 

What are you supposed to do? During the time that you want to give the child a 
power message, a respect message, and an affirmation message, the most powerful 



UNIVERSITY OF WISCONSIN-MADISON MkDomM k WUlttt 



123 



ERIC 



12: 



technique that we know is to give them the lead and to follow their lead. Affirm their 
choice of topic and explore that by following, not interrogating. You should go along 
with what they choose to do. Remember 80% of preschoolers' communication is 
nonverbal-- so often their choices can be nonverbal. Six- to eight-year-olds are 60-40% 
verbal to nonverbal, but they may start a topic with their hands. After you've read 
them a story they may reach over to the teddy bear and start patting it. Rather than 
say, "Now let's talk about the story," you could say, "Oh, you're stroking the little 
koala bear." That follows their topic initiation, and gives them respect for the choice 
that they made. Your gift to them, building on the principles of reciprocity, will 
increase their willingness to respond to your questions later. That is it— very simple! 

But this is not simple, it's very hard and very boring, because topics chosen by 
children are often boring to adults. When McDonald's son was this age they would 
walk around the block— it would take them an hour to walk around the block. Thafs 
not a very efficient use of time. He'd see a worm crawling into a hole and he'd want to 
watch it. Now, McDonald had just been teaching people about this and felt some 
obligation to do it herself, and so they sat and watched the worm crawl into the hole. 
How long do you think that took? You have to set that time aside in your mind with no 
judgments about it as an effective use of time or is it what you want to do, or anything 
like that. If the children play with the koala bear and you've just read a story to them 
about trains, you don't have to say, "I wonder if thai koala bear would like to go on a 
train?" Thaf s switching to your topic, which you can do after you've given them their 
time, but don't count it as their topic, because they know the difference. They know 
when you've switched. 

Interview Tips 

Here are ten tips that we'd like to suggest to make your time— not their time- 
more effective. Now we're into your time. You want some information from them. How 
can you get that information from them, after you have put in the time on their topic? 

The first tip is, that since you know kids like to play, you might have an object there 
that they can move about and manipulate as they talk to you* It doesn't distract them. 
They can talk just fine. If you've put in their time, they will focus on your questions. 
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Young children are mostly nonverbal; you don't have to have their eyes on you for them 
tohearyouorforthemtoansweryou. So, have objects around. You can even have big 
drawing sheets. Ifs okay to let them wiggle and squirm; they will still hear you. Ifs 
okay to have them play with an object at the same time you're asking them to talk. 

The second Up is that kids love to guess things. They especially like to guess if ifs 
easy. "Do you think I'm a man or a woman? Do you think I'm wearing glasses or not?" 
"Yes you're wearing glasses-I saw you hide them!" They will grab at the chance to 
show you their competence with the most obvious things that you can ask as you start 
It is much, much better to give them questions that you know they can answer easily, 
rather than difficult questions. They like it, and then they'll want to answer more 
questions. 

The third tip is that they like to show that you are wrong. McDonald plays a game 
with kids. For example, she says, "I'm going to tell you a story and you tell me whether 
ifs right or wrong. This is what I think, I think it snows in the summer." "Wrong!" 
they'll say. And they love it. So if you've read a story, you can do things like that. 
You can say "I've just read a story about a train going up a hill. Now do you think the 
train tracks are going through a lake or over a mountain? I think they are on a lake." 
-No, they're not, the/re on a mountain." That kind of thing. You get the information 
that you want, which is whether they were listening or not and whether they were 
understanding, and you have made it into a little game where you look stupid. (If s all 
right, you can go home and see your friends and they'll tell you you're not stupid!) This 
kind of question also allows you to get information in a nonjudgmental way without 
prying into the young child's private responses to stories. 

The fourth thing that kids like is repetition. They love rituals and repetition. The "s 
how they make connections with each other. You could read the same book fifteen times 
to the same group, one time after another. It is adults who get bored, not young children. 
You can have them listen for different things. One of the great things about home VCRs 
is that children can watch the same thing on the TV screen again and again and again. 
They love it because they can achieve mastery over it. We have always been able to do 
this with books— read them over and over. Young children love repetition because it 
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increases their power: "I know whaf s going to happen next!" That doesn't mean 
they're bored, they love it. They're not like teenagers who are in a different 
developmental stage. Little kids love repetition. On the fourth page you say, "Okay 
now, I've read this three times. I'm going to read a little bit, and I want to see if you 
know what happens next." When you put the book down, they all will know what 
happens nexi, because you've read it three times, and they love it. They love the 
mastery. 

The fifth suggestion is that they like to be powerful, and one way that they can be 
powerful is by having choices. Give them choices, don't just give them one thing that 
can happen. "I have two books today, Mary, which book would you like me to read?" If 
you've chosen enough books that you like, it won't matter to you which ones are read 
and the children feel more control and power. 

The sixth tip is that they like taking turns back and forth with you. They like 
interactional rhythm. Librarians see this when doing participation stories in story 
hours. There are different ways that you can do interaction that has rhythm to it that 
makes it more fun. Again, if s nonverbal. Repetition is the same thing, if s like word 
play. And interactional rhythm is like word play. If s taking the verbal and doing 
nonverbal things with it. 

The seventh tip is they like to be active: make them move their arms and legs as you 
listen: "How big do you think that bear was? Show me with your hands." Ifsgoodto 
have them move or act out their responses to your questions because young children are 
nonverbal. This is how they leatn. Movement helps them to remember. 

The eighth tip is kids like to know what is going on: explain what you are going to do. 
We have to be translators of the adult world, and we need to explain every step. 
Recently in teaching medical students how to do interviews, the topic was how doctors 
can better interview a child. The child tc .d them: ''Explain everything to me." And 
that was so right. We just assume that young children know when we're taking them 
down to the x-ray room, or that they know what is going on when we tato them over to 
the counter to check out the book, but they don't. We need to explain the whole process; 
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the more detail we provide, the more their sense of control increases. So you're 
respecting them just as you would a French person who doesn't know how to read 
English. You have to translate the environment for them. Going through all those 
details about the library, the rules, how to treat books, having that explained to 
respecting them 

The ninth thing is that kids like to leave a place with something in their hands. The 
library is a perfect place to go for children because you can leave with a book or a flier. 
Dentists know this, have you ever noticed? Everyone hates to go to the dentist, except 
kids, because when they go to the dentist there are toys for them to take home. The 
dentists are smart Libraries have the ability to let children take something home 
with them. 

The tenth tip is a reminder. Kids like to be respected, and they don't like fake 
behavior, and they like to be given time before you do your agenda. 

Problem Solving 

We want to talk about two little trouble shooting things. The first one is what to do 
with silence. You can generate silence with poor technique. Do not blame the child for 
your own inept interviewing skills. It is easy for us to say that children are dif flcult 
and one cannot get them to talk about anything. Think of the familiar tourist story of 
Americans speaking English to French people, and when the Frenchmen don't 
understand, the Americans just start yelling louder. What we tend to do with children 
who are silent is push them harder and try to use our power; the more we do that, the 
more they shrink into resistance. Silence should be seen first of all as an indication that 
we are not doing our job well. 

On the other hand, there are other variables which can generate silence. For 
example, children who struggle with separation and autonomy, may, because of their 
own struggles, withhold speech regardless of with whom they speak. Suspicious 
children, who have lost courage to test out new situations may keep themselves quiet to 
keep themselves safe. Children who are afraid that they will be destroyed may try to 
control the situation by feigning indifference and refusing to cooperate. Depressed 
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children, who cannot believe that anything goc 1 will happen, may just keep quiet 
because they cannot believe that something pleasant will be an outcome of their 
speech* Severely disturbed children may withdraw to insulate themselves from strong 
environmental and emotional pressures. They may be reclusive already as children and 
have emotional walls which they have built up* For these children simple techniques 
may not suffice to be effective. They may need a therapeutic approach beyond the 
skills of a librarian-interviewer* 

Some children use silence as a form of insulation* In this case one thing you can do is 
meet silence with silence and have silence be okay. Since young children are 80% 
nonverbal you may communicate effectively in silence* Silence does not obviate 
connections. And finally, some children use overtalking rather than silence, as their 
way of handling anxiety and discomfort. They achieve control of their out-of-control 
feelings by overtalking. The same intervention will slow down their talking, i.e. give 
them some power over the interactional context* 

Finally, reflecting back to making your culture a welcoming one, remember that 
having toys as well as books in your library will help facilitate your interview process* 
Puppets and toys can be stroked while children are talking to you or held while they're 
listening to you* But they can also be used in the interview itself, by allowing indirect 
communication to occur* The direct process of asking a question and answering it is one 
that people don't master until early adolescence, and even then it is problematic* It is 
not always easy for little kids to ask and answer direct questions* One of the things 
that we find is that we can increase our impact by being indirect and saying, ''What do 
you think that koala bear wants to do now?"— when you really want to know what the 
child wants to do* You can say, "Now, I read the book— do you think that koala bear 
liked this book?" Whatever the answer, you know whether the child liked it* The 
child, because he is concrete and literal, can only generate the koala bear's thoughts as 
his own* It is only when children are a little older that they can start thinking that 
someone else may think differently from what they think* If s just a trick about their 
minds, that whatever they think the koala bear thinks is what they think, because 
they really cannot imagine anything else* This indirect questioning is so much less 
threatening that children will talk and talk and talk about what the koala bear 
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thinks, and you've found out exactly what they think. There's no translation there; it 
is simply what me child thinks. 

There's one last thing that seems important to mention from the perspective of a 
mental healm professional. McDonald currently manages several projects that are 
family oriented. The projects are for children ' trouble. Rather than taking the child 
alone into therapy, the project staff see the whole family, and usually the staff go to 
the home, or have multiple family groups. One program is called "Families and 
Schools Together" (F.A.S.T.). If s a family strengthening project for kids who are high 
risk and poor learners and who are six to nine years old. These are all hard to reach, 
low income families. The parents have had terrible experiences with the schools, 
themselves. The program's goals include trying to increase their feelings of affiliation 
toward learning and school. Obviously the parents and the connections they have with 
the children with books are important. 

As you use some of these techniques yourself, you may have the chance to model for 
parents and they could benefit as weU as the library. These techniques could also prove 
beneficial for busy parents and dual-career families who like to have the library as 
part of their lives but don't seem to have time to read with their kids. You could invite 
the parents to do some of this with their child at the library. You coul- have mothers 
and daughters days or daddies and daughters days for coming in to ti* y, and you 
could have tables set up for one-to-one reading together time. Or you could read the 
story and you could have the mother with her son doing these things I've been 
describing. That would enhance both their connection with each other as well as 
letting them take the book home. 

We have presented to you why it is important to interview children, why it is hard 
to interview them, and we have argued that children are culturally different from 
adults. We have discussed what kids like in interviewing, how they think and talk, 
and what some tips are which we can use to help make connections with children in 
order to get the most out of the library. 
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Interviewing Using Micro-moments 
and Backward Chaining 



Eliza Dresang 

The interview technique called backward chaining is an interviewing method used 
to find out what an event really means to a child. It is not meant to evaluate the content 
of the event or to be a lesson; there is no hidden didactic purpose. It simply means 
"what more do you know now that you did not know before." The end product of 
backward chaining is finding out what did an event mean to you. 

Essentially, this is a communication process that has three phases: sense-making, 
micro-moments, and backward chaining. In this article this jargon will be explained 
and an understanding of the technique developed. 

Brenda Dervin, a professor of communication at Ohio State University, developed 
this process which I have modified for use with children. Dervin has a special interest 
in libraries and in 1977 published an article "Public library use, users, uses" which she 
wrote with Douglas Zweizig. She claims that one can not make sense of a user's library 
encounter just by knowing the demographic characteristics of the user. Each user has 
many such characteristics: gender, race, age, and socioeconomic class being some 
examples. Trying to determine which characteristics are important in an encounter is 
not a highly reliable way of sense making in a library situation. 

In 1980 as I began my doctoral research while developing an experimental design, I 
began to focus on attitude changes in students. Reviewing communications literature, I 
became interested in Brenda Dervin's qualitative approach to information. I decided to 
interview students using an adapted sense-making kind of interview to find out if the 
communication situation had any personal meaning for the child participating beyond 
a prescribed or anticipated meaning. 
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The results were very exciting. This type of interviewing does work with children* 
I share Lynn MacDonald's enthusiasm for learning ways to talk with children and 
understand whaf s important to them; to help them verbalize their innermost feelings. 

Why use it? In Karen Krueger's presentation, she suggested that direct feedback is 
needed from users, not just observation. How can this be done? Interviewing; a technique 
that has been tried by very few librarians, is a method that will elicit such feedback. 
It will be seen also to have relevance to discussion as a technique (presented at a later 
point in this Sourcebook). Direct feedback can discover what the user brings to us. We 
can evaluate our services better when we determine whether or how we are touching 
the personal needs of children. Without knowing all the needs of all the children this 
technique will work. Even with selective use, the information we obtain will enhance 
our understanding and improve service delivery. 

Who can apply this technique? Who can respond to it? It seems to be inappropriate 
to use it with children younger than age nine or who have not, in Piagetian terms, begun 
to move from the concrete to formal operations stage. The child must be able to deal 
with some abstractness and extended possibilities. They must have reached the stage 
of beginning to deal with abstract propositions and to think logically about them. Nine 
would seem to be the lowest possible age, and it seems possible since the interviewing 
will explore issues that directly involve the child. 

The method works one-to-one. The librarian approaches an individual child and 
asks to talk with that child about an experience. The method will not provide an 
aggregate of data such as surveys, questionnaires, or user tickets, but it will supply 
anecdotes that can supplement other data. Anecdotes, valid even when something 
happens only to one child, can reach administrators' hearts. 

This method can answer the question: "How do we know when children's services 
are good?" A child is asked to recall the major events of a program. The response could 
be used as feedback on reader's advisory services, or in a more informal way to talk 
about books. 
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When can this technique be used? It could directly follow the event. Response 
validity will not, however diminish due to the passage of time. 

Where can this technique be used? Choose an environment that is not hostik to the 
child (just as Lynn MacDonald mentioned in talking about interviewing the young 
child). Environmental details do not appear to pose a stumbling block. 

How does one adopt this technique? If s easy. A guided way of talking to children 
lets us continue what our profession is all about. You can not expect every child to attach 
a deep personal meaning to an event. The interview is not a failure if the child 
responds remembering what happened in a predictable or routine manner. 

First, ask the child to remember or recall the major events in the situation under 
discussion. Using index cards, write down each event on a separate card. Then put the 
cards in a timeline or sequence. Each card will be referred back to as you ask questions. 
Each of the events written on a card is called a "micro-moment/' and is an event the 
person being interviewed remembers from the communication situation. 

The next step is the backward chaining phase of the technique. The interviewer 
asks the child to select one micro-moment. The mterviewer asks "Did you have any 
questions (about the micro-moment listed on the card)?" After the child answers, the 
next question is "Did you get your answer?" After the child responds, the next question 
is "How did that help you?" This question is repeated and repeated until the child can 
furnish no more information. Each time this question is repeated, more self-revealing 
meaning is elicited from the child. After one micro-moment has been discussed, another 
moment is selected and the questioning process begins again. This pro cess could continue 
until all the micro-moments have been reviewed. 

I discovered the technique's usefulness as well in informal situations to move 
beneath surface meaning. If the atmosphere is right, the technique encounters none of 
the connotations associated with inappropriate prying. 



Dnung 

UNIVERSITY OF WISCONSIN— MADISON 

133 



13 



This technique of interviewing is a way to make sense out of a situation. It 
establishes a more formal structure for communication than merely talking with 
someone or observing behavior. It can illumine what meanings children attach to the 
services in the library. 
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Sampling 



Jane Robbins 



The employment of observational techniques for the purpose of evaluation often 
requires sampling because we all know that we could never observe everything that 
takes place in a library during a year! What we can do, though, is take a sample. In 
today's world most everyone believes in the efficacy of sampling. If youhave trouble 
believing in sampling, say to yourself: "Would I ever eat the entire pot of soup?" Do 
you really believe that one taste of your soup will be significantly different from 
another taste? Perhaps even a better example, if you have doubts about sampling, is 
related to a blood test. If you don't believe in sampling, I guess the next time, just give it 
all! You can count on one taste informing you of what the rest of the soup is going to taste 
like, and you can do a national sample of 1800 non-incarcerated adults and get an 
accurate measure of what a particular adult attitude is, or what a probable behavior 
will be among a group of individuals, or who will win an election, or who will support a 
library referendum, as just a few examples. 

The key for sampling is this: if you are going to take the time to do a survey of users 
(for example of parents who you think might be bringing children to the library), you 
might as well do the survey correctly . If you don't have a good sample you can conclude 
nothing from the data you collected. Since presumably you are doing the survey so you 
can collect accurate information, you want information that is v»M (If you want to 
lie, then don't bother to collect data, just say what it is you want to say. After all, 
others are not above doing it, so you can do it too.) But, if you are going <;o go into the 
process of collecting data from people, design instruments and a sample that will let 
you say: "I feel confident that this is what is going on in this library." The information 
you collect is intended as a base from which you can do good planning for the delivery of 
effective children's services. 
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Sampling can be relatively easy, but for a survey of your entire community you will 
undoubtedly want to ask someone who has a background in sampling design to design 
your sample; for example, someone from a marketing department of a large corporation. 
In order for them to help you, however, they need you to understand some of their 
language. (What you need to know can be compared to what librarians needed to know 
about computing when computers first came into libraries. Computer specialists came to 
libraries and too few librarians had sufficient basic information to support a useful 
conversation; so, the computer specialists designed the computer they believed the 
library ought to have instead of the computer which the library needed. It took rather 
a lot of false starts before librarians could gain control of the computer world in 
libraries.) Wirt\ sampling there is not a great deal you have to know in order to be able 
to have a productive conversation with a sampling designer. A sampling designer will 
ask you how much you already know about the things or people to be sampled. They 
will want to know why it is you want to draw a sample. They need you to be able to 
articulate all the intellectual reasons why you want to do it. If you can explain these 
things clearly, then someone else can draw an appropriate sample. Still, there is some 
language about sampling that you will want to know. 

First, you hear people talk about populations or universes in sampling. The 
universe is all of whatever it is you are interested in sampling. If what you want is a 
sample of children that use your library, then the universe is every child who uses your 
library. A major problem in terms of actually creating a sample is developing or finding 
what is called the sampling frame. In the example of children who use your library — 
how would you develop a list of the children who use your library? Perhaps a list from 
circulation records for the past year of every child who checked out one or more books 
could serve as the sampling frame. If you don't have access to circulation records, then 
the frame may have to be children in the school districts surrounding the library— or 
you may have to keep a list of children in the library for a month and then draw a 
sample from that list. 

Could you accurately count every child who comes into the children's room on a 
yearly basis? Undoubtedly not. But you can accurately count if you sample. Although 
we are always worried about sampling error (and samples are open to error) still there 
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is much we can do to control it. What we do in a sample is work with two different 
concepts. These concepts tell us how large the sample has to be in order to have a 
sample which only has a controlled amount of error. The two concepts or factors that 
are involved in making a precise sample size determination are tolerance and 
confidence. 

When people are talking about tolerance, they are talking about the percent 
within which the measure that they have calculated is the measure which would 
have been calculated if somehow a perfectly accurate 100% count could be taken. For 
example, one might want to take a sample of the number of children who are using the 
children's department. (To be precise, it is not the number of children but rather the 
numberof usesby children of the children's department.) The decision is to take a 
sample which would accurately represent the number of uses by children within 5%, 
which is to say a sample that would tell us within 5% more or 5% less of what the true 
number of uses are. Why within ± 5%? Because it really doesn't make a great deal of 
difference if you have perfect accuracy. (A useful analogy may be estimating the 
number of miles from Chicago, IL to Cheyenne, WY; if it is 907 miles, it could be 
estimated at 900 miles; is it important whether if s perfectly accurate?) So, by further 
analogy, if you intended to make some staffing decisions and you obtained a measure 
from a sample that said that there are 5600 uses by children of the chUdren's room in a 
year, what you would really be saying if you set your tolerance level at 5%, is that it 
could be as many as 5880 child uses, but it also could be as low as 5320 child uses even 
though the measure you obtained was 5600 child uses. You see or hear examples of 
tolerance; for example, estimates of voter preference. The sample may have obtained 
52% of the people for Dukakis-give or take 2 percentage points. This means it could be 
54% are for him or only 50% are for him 

Setting an acceptable tolerance level is a very important part of determining how 
large a sample size is needed. You should have some sense of how accurate your measure 
of child use needs to be for you to reach a management/planning decision. If the 
estimate must be within 1% you have to have a larger sample than if a 5% tolerance 
level is acceptable. If within 10% will do, you will need a smaller sample size than 
you would for 5%. Tolerance level is a major consideration related to how much effort 
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will be expended in collecting data because it is related to sample size. A sample 
designer will need to know what you consider an acceptable tolerance level; most social 
scientists select a 3 to 5% level. 

A second piece of information needed for determining sample size ha* to do with 
the confidence interval. Some of you may have heard of this even more often. The 
confidence interval is related to how willing you are to risk that the sample you select 
is not an accurate sample of the population. Any time you sample, there is a chance 
that the sample may not be accurate. You can control that chance by setting confidence 
intervals of one (1) out of 100 samples or five (5) out of 100 samples. I think often people 
lose track of just how probable it is to get an accurate sample 95 out of 100 times. As an 
example, if I >*&d a hat with $500,000 in it for your children's department and I told 
you that 95 out of 100 times when you put your hand in this hat you were going to get 
$500,000 for your children's department and li times out of 100 times you were going to 
have to give up half of your department's budget, how many of you would stick your 
hand in the hat? The probability is pretty dam good that you are going to walk away 
with $500,000 for your children's department! I think most of you would take that risk. 
By convention most social science research uses the .05 confidence interval. An .01 
confidence interval (i.e., 1 out of 100 samples could be inaccurate) would require a larger 
sample. Rarely, except in testing pharmaceuticals, is a confidence interval less than .05 
warranted. 

It would be simple for me to tell you that you now know just about all you have to 
know about sampling. Obviously there are people who spend their entire lifetime 
studying sample design, but understanding tolerance and confidence are essential to 
understanding determination of sample size. 

Another concept of interest related to sample size is sample type. The most typical 
sample and the one that forms the basis for all other probability samples is the random 
sample. A random sample requires that every item being sampled (e.g. for example 
children coining into the library) has the same probability, the same likelihood, of 
being chosen in your sample as any other item. Now you do know the minimum of 
everything you have to know to have someone else draw a sample for you. 
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Introduction to Sampling in a Nutshell 



Morris Slonim 

Simon and Schuster, 1960. Reprinted by permission. 



Everyone who has poured a highball into the nearest potted plant after taking one 
sip has had some experience in sampling. The abstemious reader doubtless has at one 
time or other pushed aside a bowl of tepid mush after swallowing a spoonful. He, too, 
has unwittingly employed a sampling technique. It is not necessary, one perceives, to 
have a graduate degree in mathematics to become reasonably proficient in sampling 
matters, in a practical sort of way. Naturally, one needs more than a stirring rod or a 
soupspoon to become a sampling expert. However, the treatment of this subject has 
frequently been at too stratospheric a level. Complex formulas and jawbreaking terms 
have been piled high and deep (whence the abbreviation Ph.D.), to the vexation and 
confusion of the reader. 

There is a tendency among some statisticians, as there is among some lawyers, 
doctors, economists and the like, to use technical terms to excess. Many laymen suspect 
that this is done to impress or confuse, or perhaps to justify a higher fee. Whatever the 
purpose, this practice is unnecessary in many cases. Such statistical terms as 
"leptokurtic," "homoscedastidty" and "interpenetrating replicate subsamples" may be 
very descriptive; however, it is mildly deplorable that statisticians have not coined 
simpler expressions for most of these technical terms. Few fishermen, for example, 
would refer to a "reticulated lattice joined at the interstices"; while the term may be 
highly descriptive, most of them use the simpler "net." 

To be sure, the origins of sampling lie in probability theory. _t is true, also, that the 
development of sampling theory requires a rather thorough mathematical training. 
Nevertheless, it is believed that the subject can be presented in a suf ficiently 
nontechnical manner to provide the so-called layman with a fair basic understanding 
of it. Sampling in a Nutshell will present the subject at the whalebone or oceanbottom 
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level Those whose number sense is sufficiently keen to enable them to count all the way 
to ten without removing their mitteas should be able to manage, while those who 
wangled a C-plus or better in high-school algebra will find it ridiculously simple (or 
vice versa). 

Essentially, sampling consists of obtaining information from a portion of a larger 
group, or "universe." This universe represents the entire lot on which some item or items 
of information are desired. The universe we desire to sample may be a large kettle of 
small fish, all the apes who wandered into apiaries by mistake last year, or the 
number of U.S.A.F. troops wearing 15-EEEE shoes. 

The circumstances in a particular problem will determine the sampling method or 
combination of methods to be employed. A variety of methods is available and the 
principal ones will be covered in subsequent chapters. 

Advantages of Sampling 

A natural and legitimate question that many readers will ask is, "What 
advantages does a sample possess over a complete count?" The answer in brief is that 
sampling often permits the executive to cut costs, reduce manpower requirements, gather 
vital information more quickly, obtain data that could not possibly be available 
otherwise, obtain more comprehensive data, and actually increase the accuracy of his 
figures in some instances. In addition, some sampling methods (as will be noted later) 
make it possible to measure the reliability of the sample estimates from the sample 
itself. Illustrations of some advantages of sampling are listed in the following 
paragraphs. 

The accountants for one airline formerly computed 100,000 fares each month on 
portions of tickets due other air carriers. They later found they could estimate the 
intercompany settlements quite accurately by sampling around 10,000 of the 100,000 
documents at a considerable dollar saving (see Chapter XVII). Some department stores 
and other large business establishments have discovered they can use samples, rather 
than take inventory of their entire stock, with considerable accuracy and a 
considerable reduction in inventory takers. 



Skmim SCHOOL OF LIBRARY AND INFORMATION STUDIES 

140 




TH need for data in a limited time period frequently makes it imperative to 
sample. Suppose an executive wants to know whether his superannuated accounts 
receivable constitute a real problem. He asks for a quick estimate of the age 
distribution of these accounts. If the company has 25,000 altogether, our e: jcutive 
might be able to get his estimate much more rapidly (and with sufficient accuracy for 
his purpose) from a sample of, say, 1,000 accounts. 

An aspirant for political office wishes to know if he has a fighting chance of being 
elected, before he announces his candidacy. To sound out each of several hundred 
thousand voters would be costly and would take an unreasonably long period to 
accomplish. He therefore has a research agency sample the opinions of a relatively 
small group. To be sure, our office seeker will probably decide to run regardless of the 
results of the sample survey. 

There are \ #f instances where sampling is the only way to get at the facts. As a 
case in point, the manufacturer of firecrackers must know whether they will explode. 
Accordingly, for a nominal fee he might permit some youngsters to light up a few 
salutes in each batch produced. If our pyrotechnics producer were not satisfied with 
anything less than a 100 per cent test, many of us would have a very quiet Fourth. 

The Census Bureau, in its decennial attempt to number all our noses, is limited in 
the number of dollars and census enumerator as well as in the time it has to complete 
this nationwide survey. As a consequence, while certain items of information are 
obtained from every U.S. resident that the head-counters can find, additional facts and 
figures are collected from a sample of one in every ten or twenty persons. The sampling 
feature makes it possible to get more detailed information than would be possible 
otherwise. 

Limitations of Sampling 

Sampling is not feasible in any situation where knowledge about each unit of a 
statistical universe is needed. Some of us would be overdrawn at the bank even more 
often than in the past if that institution, in preparing our monthly statements, were to 
use a sample to estimate total withdrawals. Neither should we expect the Internal 
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Revenue Service soon to announce that we may henceforth estimate our income and 
deductions from sample records. Consider, too, the deafening outcry from those affected 
if some employers decided to pay only a sample of their workers each payday* 

Although the point is often exaggerated by the disbelievers it is true that 
occasionally a "wiJ.d" sample may crop up, with misleading retttf t». The sampling error 
may be lai^er than expected, too, if the sample is improperly designed, or incorrectly 
carried out. There are numerous situations, finally, where the writ* to be measured are so 
rare, or their variability so great, that a sample would be a waste of money. 

Although sampling is constantly gaining greater acceptance in private industry 
and in government, many people are still skeptic** about, if not downright suspicious 
of, the accuracy of any figurts obtained from a partial rather than a complete count. In 
part, this reflects a natural raluctance to accept the evidence provided by a small 
number as correct for the entire group under study In larger part, it is due to some 
careless applications of sampling methods in the past. 

NOTE AJ though tempted, we shall not use the very much overworked example of 
a certtjn puMication that expired following a President!. 1 election poll that turned 
out disastrously. II every writer who has cited that incident as an example of improper 
aafirtpling were to contribute one dollar, the magazine might be able to resume operation. 

Sampling and Statistics 

Sampling is only oi*r component, but undoubtedly ihe most important one, of that 
broad field of scientific method known as statistics. 1 Statistics provides the tools for 
collecting, aruiJyzing and interpreting the cumbersome masses of data that most 
business and government agencies must deal with in this age of science. In many cases 
the deluge of data does not pertain to the entire universe or population under 
consideration; hence, since many of the figures at our disposal are obtained from 



The word "statistics" is used also as the pluial form to denote numerical data. 
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samples, it is certainly advisable to devote a little attention to the various methods of 
selecting samples, and of estimating such things as averages and totals from samples. 

Sampling and Probability 

There always exists the risk, in making an estimate from sample data, that the 
particular sample is not truly representative of the universe or population under study. 
This risk can be set very low by using probability" sampling methods and appropriate 
estimating techniques,* and by taking a sufficiently large sample, although it can never 
be eliminated completely. Fortunately for us, a French playboy, the Chevalier de Mere, 
in his eagerness to pin down the odds in a variety of gambling games, stimulated some 
of the most celebrated mathematicians of his day into developing a very fascinating 
branch of mathematics known as probability theory. Development of this subject made 
possible the growth of present-day scientific sampling. While sampling and sample 
estimates are possible without the existence of probability theory, knowledge of the 
degree of reliability we can attach to sample estimates would not be possible, except in 
special instances. We could, it is true, take a large number of samples of a given size to 
determine the extent of variation in an estimate, but this is very seldom practicable. 

Most of us have at least an obscure notion as to the meaning of probability. In fact, 
many who have never been inside a school of higher learning can figure the 
probabilities on a dice game more rapidly than a trained mathematician. Similarly, a 
fight fan without book learning in mathematics mentally assesses the probability that 
a specified boxer will beat his opponent before he offers 3 to 1 on his favorite. 

A rather oversimplified definition of probability may be illustrated as follows: 
If a certain event can occur in 6 ways out of a total of 36 ways that are equally 
likely, the probability of the event is defined as the ratio 6/36 (or 1/6). For, suppose we 
had a red die and a green die, the following 36 equally likely arrangements of the two 
dice could occur." 3 



2 Probability sampling and estimating techniques are discussed in subsequent pages. 

3 Assuming the dice are not loaded, shaved, rounded, or otherwise rigged for or against 
roller. 
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Fed Die Green Die 



Red Die Green Die 



Red Die Green Die 



2 
2 
2 
2 

2.. 
2 



1 

2 
3 
4 
5 
.6 
1 
2 
3 
4 
.5 
6 



3 
3 
3 
3. 
3 
3 
4 
4 
4. 
4 
4 
4 



1 

2 
3 

.4 
5 
6 
1 
2 

,3 
4 
5 
6 



5 
5. 
5 
5 
5 
5 
6. 
6 
6 
6 
6 
6 



1 
2 

3 
4 
5 
6 
,1 
2 
3 
4 
5 
6 



Of these 36 ways, 6 are seen to give a total of seven dots (the connected pairs). From 
our definition above, the probability of rolling a seven with two dice is seen to be 6/36, 
or 1/6. Similarly the probability of getting a total of two dots on a throw of two dice 
would be 1 /36, because the two dots (or "snake eyes") can be thrown in only one way. 

Similarly, we can readily determine that the probability of getting two heads in 
one toss of two coins is 1 /4, or 25. 



Coin 1 


Coin 2 


Head 


Head 


Head 


Tail 


Tail 


Head 


Tail 


Tail 



NOTE: If the probability that an event will occur is 1/4, then the probability it will not 
occur is 1 minus 1 /4 (or 3/4). A common way of expressing probability is in terms of "odds in 
favor" or "odds against " The odds are expressed in terms of the numerators of the two 
probability fractions. In the present instance the odds are 3 to 1 against occurrence of the two 
heads. Similarly, if a man has two tickets out of 100 in a raffle, the probability of winning is 
2/100; of losing, 98/100. The odds against winning are therefore 98 to 2, or 49 to 1 . 
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When we begin to talk about rolling 16 dice, or tossing 37 coins at a time, we must 
resort to mathematics rather than attempt to enumerate all possible ways of 
occurrence. It should also be stressed that probability theory delves into a number of 
other types of problem, many of them highly complex in nature. 

Very often in elementary probability, the answers to problems are not at all what 
one would expect intuitively. Following are two simple examples that will illustrate 
this point 

l.The "Birthday Paradox" 

Readers may find it diverting or even profitable to learn about the "Birthday 
Paradox." This concerns itself with the odds that at least two persons in a group of 30 
hav ! the same birthday. The uninitiated layman usually estimates (following some 
celeritous cerebrations), that the odds are around 11 to 1 against the occurrence of such 
an event. His reasoning is along the following lines: There are 365 possible birthdays 
and only 30 people; hence the probability is 30 in 365, or approximately 1 in 12, of a 
match in birthdays. Actually, the odds are better than 2 to 1 that at least two of the 
thirty people will have the same birthday. 

In order to check the theoretical odds in an actual case, the author selected, at 
random, page 188 in the Air Force Register of January 1, 1958, and tested the first ten 
groups of 30 names. Nine of the ten had at least two officers with the same birthday. If 
a large number of groups of 30 each were examined, approximately 7 out of 10 would 
have at least two persons with the same birthday. The dubious reader can check these 
odds by sampling several different groups of 30 people from such publications as Who's 
Who, Men of Science, etc. 

Experience, as well as mathematics, in this instance discloses the error in one's 
intuitive feeling that the occurrence of multiple birthdays in a group of 30 people is 
rare. One way to explain the rationale of this apparent paradox is as follows. Picture 
the 30 people lined up in a row. Number One states his birthday. The remaining 29 then 
compare their birthdays with his. If there are no matches, Number Two then announces 
his birthday. The remaining 28 now have a second chance to compare their natal dates 
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with that of Number Two. If, again, none match up, one or more of the remaining 27 may 
still duplicate Number Three's birthday. So on down to Number Twenty-nine, whose 
birthday may still be the same as Number Thirty's. Each of the 30, therefore, has 29 
separate chances of matching his birthday with another's. 

Another way to explain the variation between the average person's intuitive guess 
and the true probability was stated very aptly by Professor Jerome Cornfield of Johns 
Hopkins University. He said, in effect, that the average person sees the problem as 
"What are the odds that any one of the other 29 has the same birthday as mine?" 
whereas he more properly should ask, "What are the odds that any one of the 30 has 
the same birthday as any other one of the 30?" 

The formula for computing the probability (denoted by "p") in this problem is: 

{30 terms) 

p-1 365 X 364X...X .136 
365 X 365X ...X 365 

{30 terms) 

As one might expect, the odds in favor of multiple birthdays drop as the number of 
people decreases. For 23 people if s about a 50-50 bet. On the other hand, the odds rise 
sharply as the size of the group increases. Now that there are fifty states in the Union, 
and 100 U.S. Senators, the skeptical reader might like to use the same formula (but 
with 100 terms in each part of the fraction) to calculate the odds tltat at least one 
Senator has the same birthday as some other Senator. The odds (and we hope you 
didn't take 3 to 1 against) are more than 3,000,000 to 1 that at least two of our senior 
solons can split the same birthday cake! 

2. The Derby Problem 

Ten men wearing size-7 black derbies attend a party. On departing, each reaches 
for the nearest derby in the pile. It turns out that the odds are about 63 to 37 that at 
least one of the ten will retrieve his own hardtop. Suppose instead that there had been 
10,000 men in the same situation. Of a number of people queried by the author, most 
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thought that the adds would then be much greater that at least one man would retrieve 
his own headpiece. Only a few thought the odds would be less. As a matter of fact, the 
odds are almost identical, whether there be ten or ten million men in the problem. 
Although the solution is far from obvious, the probability of this particular 
happenstance is: 

p = 1 - 1 + 1 • - 1 + etc., etc. 

1 2X1 3X2X1 4X3X2X1 

There will be as many terms in the equation as there are derbies. Since the tenth 
term is less than .0000003, and since every even-numbered term is negative, it is easy to 
see that the probability will change hardly at all by the time we get beyond ten 
derbies. 



Sampling Error 

Whenever we estimate something from a sample, it is most unlikely that the 
sample estimate will be exactly the same as the figure obtained by making a 100 per 
cent count. The difference between the two is the sampling error, if both sample data 
and 100 per cent data are gathered using identical methods. 

When the data-gathering methods are not identical, much of the difference 
between sample estimate and 100 per cent count may be due to the variation in these 
methods. For example, suppose our task were to determine the total weight of 5,000 fish 
caught by 1,000 fishermen on a given day. We could obtain the required ichthyological 
poundage by getting each of the 1,000 fishermen to write down his estimate of the 
weight of each of his fish, then adding the 5,000 figures. If next we weighed a sample 
of 500 of the 5,000 fish and multiplied by 10, it is highly probable that there would be 
a considerable difference between the two totals. Moreover, in all likelihood only a 
portion of the difference would be due to the act of sampling. The lion's share of the 
spread would undoubtedly be attributable to the somewhat biased method of collecting 
the figures in the complete enumeration. Suppose that instead of weighing 1,000 fish we 
had taken a sample of 1,000 of the 5,000 figures jotted down by the anglers and 
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multiplied the sum by 10. The difference between this estimate and the sum of the 5,000 
figures would be the sampling error. Similarly, the difference between the sample 
estimate and the 100 per cent count, if both were obtained by actually weighing the 
fish, would also be a sampling error. 

Accuracy versus Precision 

As a matter of incidental intelligence, it is customary in the profession to refer to 
the precision, rather than the accuracy, of a sample estimate. The accuracy of an 
estimate refers to the degree to which it approximates the true figure. The precision 
measures the degree to which it approximates the figure that would be obtained from a 
100 per cent count if identical methods of data gathering were used. Anyone who is of 
the notion that a 100 per cent count automatically provides the true figure should try 
counting and re-counting a heap of boiled peas the size of a dunce cap until he gets the 
same total twice. 

Probability Sampling 

To be sure, a sample always carries with it a sampling error. However, many are 
unaware that for certain types of samples the degree of precision desired from the 
sample estimate can be stipulated in advance. Such samples are called probability 
samples or "scientific" samples and are characterized by the fact that the probability 
of selection of each unit is known. 

For example, if we numbered twenty identical plastic chips from 1 to 20 and mixed 
them carefully before selecting one, the chance (or probability) of selecting any one of 
the chips would be one in twenty, or 1 /20. If we added five more chips containing the 
number 7, the probability of selecting a 7 would be 6/25, while for any other number it 
would be 1/25. In both cases the probability of selection is known in advance, although 
the probabilities, in the second case, are not equal for all numbers. Fortunately, in a 
myriad of practical instances, whether dealing with accounts receivable, shoe sales, 
inventory lists, military pay records, factory workers, cans of beans, aircraft or any of 
very many other items, the probability of selection of each sample unit is known in 
advance. When this is so, a probability sample can generally be selected. 
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Sampling with and without Replacement 

The number of units in our statistical universe may be finite. That is to say, they 
may be limited in number, as the number of pizza pies a man with an alimentary 
ailment can consume in a year. At times, our universe may consist of ^infinite 
(indefinitely large) number of units. The number of "comic books" to be published in the 
limitless future (a cheery prospect) is an example. Suppose we had a bowl of a 
thousand plastic chips numbered 1 to 1,000 from which we plan to select a sample of one 
hundred. In selecting the sample, we could put aside each chip we drew before reaching 
into the bowl for the next one. Such a sample, obviously, would contain no duplicates. 
This is an example of sampling without replacement, which is the same thing as 
sampling from a finite universe. If, on the other hand, we replace each chip in the bowl 
before drawing the next chip for our sample, the process is termed sampling with 
replacement and simulates what happens when we sample from an infinite universe. 

It develops that the formulas for calculating the reliability of sample estimates 
are more complex when the sampling is from a finite universe (i.e., without 
replacement). If the universe is rather small, or if the sample size represents a 
significant portion of a large universe (say 10 per cent or more) it is wise to use the 
formulas for sampling without replacement. In many practical situations, happily, the 
universe is so large and the sample size so small percentagewise that we can treat our 
universe as infinite. This simplifies the calculations that must be performed. 

Tolerance and Confidence Limits 

Suppose a grommet manufacturer wished to estimate from a sample the average 
pay of his production workers. He could specify in advance the degree of precision he 
would like in his estimate, if a probability sample were used. The precision 
specification consists of two elements. Our manufacturer might specify, for example 
that he would like his estimate to be within 5 per cent of the figure obtained by a 
complete count. This error limit is called the tolerance specification. And, since he is 
dealing with samples, he must assume a certain degree of risk that an occasional bad 
sample may give an estimate less precise uian he has specified. 
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This risk can be set at any desired level. Our manufacturer can express the risk 
specification in terms of betting odds, e.g., 99 to 1 that the error will not exceed 5 per 
cent; or in terms of the number of samples, e.g., that the error will be 5 per cent or less in 
99 out of 100 samples. This expression of risk is called the confidence specification. Both 
tolerance and confidence specifications of a probability sample must be stipulated 
before the required sample size can be determined. 

It should be stressed that the manufacturer, not the statistician, is the one whom 
we ask to set the error and risk specifications. In general, the level of precision desired 
from the estimate is a matter for management rather than the technician to determine. 
For management is best able to assess the practical impact of variations in precision on 
its operations. 

Some Additional Arguments for Sampling 

It is very interesting to note that the precision of a particular probability sample 
can be estimated from the sample itself. This most remarkable property of probability 
samples provides a potent argument for using such samples in place of judgment or other 
nonprobability samples. 

Furthermore, many who insist that the only accurate way is to make a complete 
count overlook the fact that there are numerous sources of error in the basic data and 
that a 100 per cent count can be highly erroneous as well as nearly impossible to 
achieve. Consider, for example, the very likely understatement of the total population 
in certain mountainous regions where the haze makes it difficult to distinguish a census 
enumerator from a revenue agent. In fact, a sample can sometimes yield more accurate 
results than an attempted complete count because the sources of error can be controlled 
much more effectively when only a small number of items are to be examined. 

An example of this is an estimate of the value of the inventory on hand for some 
500,000 Air Force consumption-type items. The estimate was derived from a sample of 
some 60,000 items, and was about 8 per cent less than the total value obtained from a 
complete count of the half-irillion items. However, basic records were checked 
thoroughly for the 60,000 items in the sample and many revisions were made. No 
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similar check was made for the remainder of the 500,000 items. Consequently, there is 
an excellent possibility that the sample estimate is closer to the true value than is the 
complete count. 

No doubt about it-sampling is here to stay. Judging from the experience of the past 
decade, the use of sampling will continue to expand at a rapid pace. It is already being 
used to a moderate extent in many organizations and seems very likely to assume much 
greater importance in the future. Sampling is an extremely useful tool with which 
every individual should have at least a nodding acquaintance. 
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Questionnaires 



Adapted from part of a speech by Jane Robbins 

Many people have designed a questionnaire but few have thought it a successful 
effort; yet, just because it didn't work the first time, because you learn so much in the 
process of designing a questionnaire, you should do it again. The second time around it 
will be a lot more successful. We still have the belief that anybody can write a 
questionnaire, but it is the hardest data collection method around. Even lots of 
experience doesn't necessarily improve the first version. You must let a sample of the 
people who are going to answer the questionnaire let you know if they can complete it 
and if you are really getting answers to the questions you believe you are asking. 

There are many kinds of techniques related to questionnaires. One of the things we 
know is that if you telephone respondents about 20% more people tell us they use the 
library than those for whom we have evidence of use. Why? Because people don't feel 
good about telling us that they don't use the library! It may be because Miss Schmutz in 
2nd grade made them feel like they should be library users; or it may be that they feel 
it doesn't sound very cultural to say they don't use the library so they just say they do. 
Because we know about this, one of the things you will want to do is to design questions 
that allow people the opportunity to say they don't use the library. You could use a 
question that starts out: "There are a large number of community services available for 
use. Some people prefer parks to baseball stadiums, some prefer bowling alleys to 
libraries. We would like to know about one community service available: Have you 
used the library in the last year?" When a question is stated in this permissive fashion 
a respondent has some reason to say, Tm the bowling type, not the library type." How 
you ask a question should depend on what information you want. If what you want is to 
tell the city council that 80% of the people in town use the library even when you know 
thaf s not really true, you can design a questionnaire to get you that information. (But 
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why not just tell them that 80% use the library, rather than designing a questionnaire 
to get the information?) 



If you're going to design a questionnaire, it is important to share it with other 
librarians who have designed questionnaires before sending it out for a pretest, 

Questionnaires to be sent to adults are fraught with difficulties. Certainly 
designing questionnaires to be filled out by children is doubly difficult Level of 
language, ease in answering, avoiding interference of adults in the child's progress in 
answering are all concerns in designing questionnaires to be answered by children, 
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Construction of the Questionnaire in Survey Research 



Barbara B. Moran 

Reprintedfrom Public Libraries Summer 1985 p. 75-76. 

The survey is a commonly used method in library research, and the written 
questionnaire is a common mechanism for collecting data. The questionnaire is 
relatively inexpensive and can be used to obtain information from geographically 
dispersed persons. Unfortunately, it sometimes produces a low response rate, and it does 
not provide a way to furnish additional information about questions that are confusing 
to the respondent. This article examines ways to design questionnaires that will elicit 
the required information. 

There are at least two elements necessary to ensure a good questionnaire: 
(1) construction of good questionnaire items and (2) design of a pleasing layout for the 
questionnaire. Each of these will be examined in turn. 

Survey questions may be classified as unstructured or structured. Unstructured 
questions (open-ended questions) allow the respondent to reply freely without having 
to select a provided response. Unstructured auestions are especially useful in 
exploratory research where not enough is known about a subject to allow the researcher 
to provide meaningful alternatives. Unstructured questions provide rich data, but 
responses are difficult to code. 

Structured (dose-end) questions are characterized by a group of fixed responses. 
Survey participants can choose among answers designed to reflect various views, 
beliefs, or feelings. Answers to structured questions are ''pigeon-holed'' into slots. They 
provide uniform response and are easier to analyze. For this reason, most survey 
researchers tend to prefer s*. •uctured questions. 



/ 

/ 



UNIVERSITY OF WISCONSIN-MADISON Moran 

155 

IRS 



It is hard to write a good questionnaire. Because the quality of the instrument has 
such an influence on response rate, a survey researcher must write and rewrite items 
until the instrument is as clear as possible. One of the reasons that it is hard to write a 
clear questionnaire is that the person doing the survey is usually deeply involved in 
the subject being studied. The same thing cannot be said of the respondents. What may 
be crystal clear to the researcher may not be clear at all to persons who have given 
little or no attention to the topic. 

A survey should be both clear and concise. Every questionnaire item should measure 
a specific attitude or collect a specific bit of data. There is a temptation to add 
superfluous items to a questionnaire, but resist the temptation. 

There are many books available to aid in the construction of good questions; some 
are in the bibliography. The following are suggestions on questionnaire construction: 

1. Make questions as short as possible. Make them easy to read and answer. 
Eliminate repetition. 

2. Avoid questions with negatives; they are easily misinterpreted. Many 
readers just will not see the negative. In the question, "Do you think the 
library should not charge fines on overdue books?" many will overlook the 
word not, 

3. Do not use slang, jargon, and technical terms. For instance, do not ask 
whether the user prefers a KWIC index to a conventional index. The 
occasional library user will not understand the distinction. 

4. Avoid "double-barreled" questions; i.e., questions that require two answers, 
e.g., "Do you think that the library should circulate films and art prints?" 
If the respondent feels that the library should circulate one and not the 
other, the question as written does not provide any option for that 
viewpoint. It should have been divided into two questions. 

5. Be certain that the options provided for structured questions are 
exhaustive and mutually exclusive. The options should provide a place for 
all possible responses (this is often done by providing an option for "don't 
know," "undecided," or "other, please specify"). The respondent should not 
feel compelled to select more than one response. 

6. Make questions as impersonal as possible. 
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7. Avoid biased or leading questions where a "right" answer seems to be 
implicit to a particular item. 

8. Sequence questions from the general to the specific, and begtojrtth 
mterestir^ nonthreatening questions. Questions on topics that might be 
cohered sensitive or personal are generally better placed near the end. 

Writing good items for the questionnaire is much more difficult than is usually 
imagined. As mentioned earlier, the researcher is often unable to distinguish ambiguous 
questions because of deep involvement in the subject. When the survey is in a rough 
draft form, it must be pretested to ensure that the questions are unambiguous and that 
the directions are sufficiently clear to allow respondents to supply the data needed. 
Every survey should be pretested before it is in final form. 

Pretesting usually involves two processes. The first is to send the survey to a group 
of people as similar as possible to the survey sample. Often the pretest is sent to 
individuals who were in the original population, but who were not chosen in the 
sample. The second part of the pretesting involves letting other people read the 
survey, fill it out, and comment on it. Friends, relatives, and colleagues can all be 
utilized. The more people who pretest the survey the better, since different protesters 
point out different possible flaws. The survey instrument is modified as a result of the 
pretest, and the final draft is produced. 

When items in the questionnaire are in final form, the actual survey instrument is 
designed. Here, one concentrates on the appearance and the arrangement of the survey 
form itself. One of the most important contributing factors to a high response rate is 
appearance. Often respondents seem to decide whether they will participate or not 
based on the appearance of the survey. 

Much research has been done on what makes a survey appear attractive to a 
possible respondent. Most results corroborate what is common sense. The questionnaire 
should be uncluttered, with a lot of "white space" to make it appear less formidable. 
Squeezing as many questions as possible onto a page n es the survey shorter in pages, 
but the clutter may result in overlooked questions or in respondents deciding not to 
participate. Always leave enough room for answers to be written out. 
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Mail surveys and the correspondence to accompany them should be reproduced in 
the most professional manner possible. Ditto and mimeograph questionnaires should be 
avoided. They are hard to read and discourage recipients from completing them 
Offset printing is an ideal way to reproduce questionnaires, but good photocopying can 
be used. 

Questionnaires are no longer restricted to black ink on white paper. Research has 
shown that questionnaires that use colored inks and papers often result in a higher 
response rate that those done in black and white, presumably because they are more 
eye-appealing. 

Good questionnaire design is not the only requirement for quality survey research, 
but it is an important one. The sources in the bibliography include more information 
about all aspects of survey research. 
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Examples of Questionnaires Designed to Deal with 

Children's Services 



A number of questionnaires are provided on the following pages to serve as 
examples or models to be adapted. 

The first three examples, "Public Program/Group Visit (Preschool-Third Grade) 
Parent/Teacher/Caregiver Response Form/' "School Site Visit Audience Response 
Form," and "Public Programs/School Visits to Public Library Audience Response Form" 
were adapted from evaluation forms used by the Montgomery County (MD) Department 
of Public Libraries. The "Children's Services Diagnostic Survey" was adapted by Jane 
Robbins from a survey prepared fot the St. Tammany Parish (LA) Public Library. 

The survey form displayed on page 168 is taken from Ellen Altaian, Ernest R. 
DeProspo, Philip M. Clark, k Ellen Connor Clark, A Data Gathering and Instructional 
Manual for Performance Measures in Public Libraries, Chicago: Oberon Press, 1976 and is 
reprinted here by permission of Philip Clark. This "user ticket" is intended to be 
printed on lightweight stock. Numbering each ticket on the back allows a count of the 
number of tickets distributed. One ticket is handed to each person coming into the 
library (or into the children's room). Before handing the ticket to a visitor, a staff 
member draws a line through the fifteen-minute interval that corresponds to the 
arrival time. 

Each visitor leaving is asked for the ticket received upon entering. The departure 
time is marked in the appropriate fifteen-minute interval. 

The questions asked on the user ticket can be changed as appropriate for the library 
or the clientele. 
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Public Program/Group Visit (Preschool-Third Grade) 
Parent/Teacher/Caregiver Response Form 

We would appreciate your comments about our program to help us evaluate its worth 
and possibly change it, if needed, to be more appropriate to your needs. 



Program: 


Date: 


_/ 


./ 




My (or my school/agency) Zip Code: 










Please circle the most correct answer for you: 










1. The overall delivery of the program was 


excellent 


good 


fair 


poor 


2. The involvement of the children was 


excellent 


good 


fair 


poor 


3. The stories/materials chosen were 


excellent 


good 


fair 


poor 


4. The activities chosen were 


excellent 


good 


fair 


poor 


5. The length of the program was 


excellent 


good 


fair 


poor 


6. Ihe time /day of the program was 


excellent 


good 


fair 


poor 



7. 1 learned about the program from: 

Library Events Listing L ibrary flyer/poster 

Library staff Radio: 



_Newspaper: Friend 

_TV: Teacher 

Cable: Other: 



My child/class is male female. (If class, please give numbers.) 

We have have not used the library before. 

This is is not the first library program I/we have attended. 

I do do not have a library card. 

(If parent) My child does does not have a library card. 

I would would not recommend this program to a friend. 

Please use the reverse side of the form to share anything you feel would help us 
improve this program, or to suggest new programs we might do for you. 
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School Site Visit Audience Response Form 



School: Date: / / 

Grade: Zip Code: 

Please circle correct information: 

1 . jam male female 

2. I have a public library card. yes no 

3. I have used the public library since last summer. yes no 

4. This is the first time a public librarian has spoken 

to my class. yes no 

5. I would recommend this program to a friend. yes no 

Please use the bottom of this form to share anything you feel would 
help us improve our visits to your school, or to suggest new ideas for 
programs we might do if we visit again. 
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Public Programs/School Visits to Public Library 
Audience Response Form 



Program: 
Library: _ 



Please check correct information: 
I learned about the program from: 
Library Events Listing 
L ibrary staff 

Newspaper: 

TV: . 

Friend 



Date: 



/ / 



My Zip code: 



Library flyer/poster 

Radio: 

Cable: 



Teacher 



Other: 



lam _male female. 

My age is between: 

5-10 41-50 

11-20 51-60 

21-30 61-70 

31-40 71-80 

I do do not have a library card. 

I have have not used the library before. 

This is is not the first library program I have attended. 

I would would not recommend this program to a friend. 

Please use the reverse side of this form to share anything you feel 
would help us to improve our public programs, or to suggest new 
programs we might do for you. 
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Children's Services Diagnostic Survey 

Adapted from a survey designed by Jane Robbins 



Please take about 15 minutes of your time to answer this questionnaire in order to help the 
library plan for its children's services over the next several years. 

A. General Information 

1. How are you affiliated with the Library? (Circle number of corrert response.) 
(1) Employee (2) Trustee (3) Representative of Group Users 

2. For how many years have you been associated with public libraries? 

(1) Under 5 years (2) 5 to 10 years (3) 11 years or more 

3. How long have 3'ou lived in [community name]? 

(1) Under 5 years (2) 5 to 10 years (3) 11 years or more 

(4) NA, don't live in [community name]. 
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B . Most Important Library Activities, 9 xvices, Groups 



4. ACTIVITY AREAS 

If you had $1000 to spend on children's library activities, how would you 
distribute your $1000 among the areas listed below? 

(You do not have to spend money for each iirea, but you must spend the 
whole $1000. For example, someone might wish to spend their $1000 as 
follows: $0 culture, $300 education, $400 information, $300 recreation.) 



Culture (including appropriate programs and 
materials; e.g., art books, prints, award winning 
children's literature, classical records, displays, 
exhibits, etc.) 



c. Information (staff activities in the location of 
information and materials, information questions, 
information programming, reference collections, 
etc.) 



d. Recreation (programs for entertainment, print and 
non-print materials for leisure use.) 



b. Education (including support of school programs, 
i.e., homework support for students and curriculum 
support for teachers; self-teaching materials, 
learning programs for the educationally 

disadvantaged, the gifted, etc.) $. 



Total $ 1000 
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SERVICE PRIORITIES 



ERIC 



a. 



a. 



What amount of total effort and resources should be devoted to each of the 
following services? (Assign between $0 and $1000 to each of the services listed. 
Must total $1000.) 

Information services (including reference, referral, 
and other STAFF services aiding children and those 
working with children in the use of materials or 

the securing of information.) * 



b. Materials (including the selection, acquisition, 
processing, and circulation of print and non-print 
materials, and their use by children and those who 

work with children.) - 

c. In-library programming (including all programs, 
exhibits, and displays for children and those who 

work with children.) *- 

d. Out-of-library services (including bookmobile 
service, books-by-mail projects, programming outside 
the library such as visits to schools, speeches or 
programs for special groups, deposit collections, 
visits to the homebound, etc.) 



Total 



SERVICE GROUP PRIORITIES 



$_1Q0JL 



What amount of total effort and resources should be devoted to each of the 
following service groups? (Assign between $0 and $1000 to each of the service 
groups listed. Must total $1000.) 



Children through preschool age. »- 

b. Children 5 to 8. $ - 

c. Children 9 to 13 $ - 

d. Special groups of children (special services for 

the disadvantaged, gifted, handicapped child, etc.) $. 



Total 



uom. 
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7a. What three (3) added services do you think would most increase use and/or 

satisfaction with our children's services? (Please read through entire list before 
replying.) CIRCLE THREE ONLY. 

More weekend hours of service 1 

More weekday hours of service 2 

More evening hours of service 3 

Added facilities for children's services 4 

Expanded bookmobile service 5 

More books for preschool children 6 

More books for children 5 to 8 7 

More books for children 9 to 13 8 

More records, cassettes, videos, CD's and 

other non-print materials 9 

More magazines for children 10 

More staff in children's services department 11 

More reference and information materials 12 

More library programs for children 13 

More special services for handicapped, homebound and 

confined children 14 

More reference and information services to children 15 

Additional parking space 16 

Purchase of duplicates of popular children's materials — 17 

More publicity about children's library services 18 

Renovate children's facilities 19 

More materials and services for adult professionals 

who serve children 20 

More materials and services for parents 21 

More equipment for children to use in the library, such as 

microcomputers, VCR's, etc. 22 

Additional bike racks 23 

Other (Please identify) 24 



b. Of these three, which ONE do you think would most increase use and /or user 
satisfaction of the Library? (Enter the number of one of the items you circled 
above in part "a" of the question.) 



(Please enter number only) 
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8a If decreased funding forced reductions in library expenditures for children, which 
three (3) of the following actions should be considered? 
CIRCLE YOUR TOP THREE RECOMMENDATIONS. 



1 

2 



Reduce number of books purchased 

Reduce number of duplicate copies purchased 

Reduce number of children's magazine subscriptions 3 

Reduce records, cassettes, videos, CD's and other 

non-print materials 

Eliminate bookmobile services "~" jj 

Close one day during week " 6 

Close some weekday mornings 

Close some evening hours 

Other (Please identify) 



4 



b Of these three, which ONE do you think would least decrease use and/or user 
satisfaction of the Library? (Enter the number of one of the items you circled 
above in part "a" of this question.) 

(Please enter number only) 
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A 

1. Male 2., , female 


Staff 
Use Only 

:30 3 


OCCUPATION 


:45 4 

:30 7 


C. IF YOU ARE NOW A STUDENT 
PLEASE CHECK BELOW 

1. Part Time 2. Full Time 


:30 11 

;45 i* 


1. Nursery 4. H.S. (10-12) 


12:00 13 
:15 14 


2. Elem.(K-6) 5. College 

3. J. High (7-9) 6. P. Qrad 

7. TechnlcalA/ocatlonal 
2. Other (specify) 


-« 

1:00 17 

il IE 
:4S 20 
2:00 21 




:30 23 
:45 24 


D. Did you ask a staff member for help today? 
1. Yes 2. No 


- ;99 27 

:4S 26 
4:00 29 


E. Did you obteln the library material 
you wanted today? 

1. Yes 2. No 3. Partially 


:15 30 

HHr 


If not, why not? 


5:00 33 
:15 34 
:30 35 
:45 36 


F, Did you receive the library service 
you expected today? 

1. Yes 2. No 3. Partially 
If not, why not? 


6:00 37 

:45 40 

:30 43 
:45 44 


fFor further ttimmante mca nthor clrio \ 

PLEASE RETURN TICKET 
AS YOU LEAVE LIBRARY 

THANK YOU! 


._fl;QQ 45 

:15 46 

;?o 47 

:15 50 
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Introducing an Environment Rating Scale 
for Public Library Children's Services 



Holly Willett 

Background 

In 1982 1 was a doctoral student in the library school at the University of North 
Carolina at Chapel Hill. I held a graduate assistantship to manage a one-person 
library which was a books-by-mail service for day care directors and educators. The 
tiny library was part of the Day Care Technical Assistance and Training System 
(DC/TATS), which in turn was part of a large organization devoted to the study of 
children, »he Frank Porter Graham Child Development Center, at the University. The 
people that I worked for-Thelma Harms, Debby Cryer, and Dick CM ford- had been 
researching and thinking about day care for many years, and one u their most recent 
publication? (at that time) was the Early Childhood Environment Rating Scale 
(ECERS) (Harms & Clifford, 1980). 

ECERS combines the theory and practice of day care into a rating tool for day care 
workers and researchers. That is, Harms and Clifford applied child development 
theories to physical facilities, activities, program design, and staff behavior towards 
children and parents, realizing that all of these elements contribute to the 
effectiveness of day care in stimulating optimal development of young children. 
ECERS was designed to help day care providers answer such questions as "How well 
and to what extent are we doing what we should be doing and how well does our 
environment fit our purpose?' ECERS has been tested for its reliability and validity, so 
researchers can use it to examine groups of day care centers or random samples of day 
cares. Harms, Qif ford, and Cryer have expanded their work and have recently 
published rating scales for infant day care and home day care. 

Among ECERS's many virtues are its clarity and ease of use. Each item on the scale 
is carefully explained, as is the rating system itself. I believed that such a scale would 
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be helpful to children's librarians engaged in remodeling an existing children's room, 
building a new library, or making a major change in the activities of the children's 
department. An environment rating scale for children's areas could also be used by 
library consultants working with individual libraries and library researchers studying 
library design effectiveness. The concept upon which such scales are designed is called 
"synomorphy," the fit between a program and its milieu. Synomorphy is based on the 
notion that institutions will come closer to their goals if their environment— physical 
facilities, equipment, spatial design, staff behavior patterns, etc. — suits the program 
(Gump & Ross, 1977). If the milieu and program don't suit each other, either or both 
may change, and in the process of changing, the institution's effectiveness may be 
affected, positively or negatively. 

For example, in the 1970s many schools were built in an open plan with no walls 
between classrooms and the school library in the center of the building at the 
crossroads, so to speak. If you go into open plan schools today, you often find that 
various kinds of barriers have been used to separate classrooms from each other and 
from busy traffic areas. Teachers discovered that students were distracted by the noise 
and movement going on in other areas around them, and so, teachers have constructed 
"walls" of furniture and space dividers. In one middle school in Madison the library's 
bookstacks have a solid back and they have been placed around the periphery of the 
library to form walls that the architect did not provide. 

A mismatch between environment and program may also cause those involved to 
change the program. In some open schools, the staff have changed their activities 
rather than the physical facilities. At a local open plan elementary school, classes are 
reminded to walk past the library in silence, which reduces noise, and the librarian 
feels that both students and teachers are more aware oi the library than they would be 
if it were self-contained. 

Many such examples could be given for public library services for children. A very 
obvious one is the library which does not have a separate room for activities so that 
storyhours and other events are carried out in the children's room itself. To avoid 
disturbing patrons using the children's room for reading or study— purposes often held to 
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be more appropriate than a film or craft activity-few programs are provided or 
programs are scheduled at times when the room is less used. These times might not be 
the ones which would attract a sizable audience, but the program has to be changed to 
suit the environment. Library buildings more than fifteen years old provide examples 
of modifying the environment to suit the program: Considerable time, money, and 
planning are required to add electrical outlets and equipment to operate contemporary 
materials such as audiovisuals and computer. 

Development of the E-scale 

ECERS helps day care professionals identify areas where either the program or 
the physical facilities need to be modified in order to come closer to providing an 
optimal environment to foster the development of the whole child. Although they do 
not often phrase it in quite this way, enhancing the development of the child is a goal 
that children's librarians share with child care providers. Because the means and 
methods librarians use differ from child care, ECERS could not be used as is; it had to be 
adapted. Day care must be far more comprehensive than libraries in the things it 
provides for children, including food and sleep as well as educational experiences. On 
the other hand, children's services in public libraries serve a broader range of ages than 
most day care, sometimes encompassing neonates up to fourteen year-olds. Librarians 
are somewhat specialized in their concerns, believing in the efficacy of library services 
and resources to enhance child development; particularly we believe in the value of 
reading in the lives of children. Therefore, we have some concerns that are not shared 
with day care, such as reference service and lending materials. 

The emphasis that ECERS places on child development and the environment can be 
very helpful for librarians, nevertheless. Children's librarians are sometimes accused 
of being more interested in children's books and other materials than in children 
themselves. A survey of children's librarians in California found that only 20% of 
them had taken a course in child development, for example (Graver & Moore, ca. 1981). 
A client-centered environment rating scale for public library children's services can 
help staff balance their concerns for materials with a greater awareness of the needs of 
children and adults who live and work with children. 
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A new evaluation instrument for public libraries should also take into account the 
work on evaluation that has emanated from the Public Library Association since 1973. 
The Association has produced documents on the planning process, the mission of public 
libraries, output measures, and role-setting. All of these mateiials represent a wealth 
of professional expertise that should be incorporated into new tools. For the purposes of 
the Environment Rating Scale for Public Library Children's Services (E-scale), it has 
seemed most reasonable to use the eight roles described in Planning and Role Setting 
for Public Libraries (McClure, et al., 1987). 

Each section of the E-scale indicates the roles that seem to be most reasonably 
related to that section, in my judgment. While only one of the roles from Planning and 
Role Setting— Preschoolers' Door to Learning— specifically refers to children, any of 
the eight roles can apply to children's services. For example, the section of the E-scale 
that rates Educational Experiences for Children is related to four roles: Formal 
Education Support Center, Independent Learning Center, Preschoolers' Door to Learning, 
and Reference Library. Some parts of the E-scale I considered so basic, such as Safety 
and Comfort, that the Related Roles note simply says "All." 

In developing the individual items to be rated, potential items were submitted to 
an advisory panel which consisted of all members of tk nstitute advisory panel plus 
others chosen to give a wider geographical representation. The other E-scale advisory 
panel members were Therese Bigelow of the Hampton (VA) Public Library, Gale Eaton 
of the University of Rhode Island, Anne Osborn of (he Riverside (CA) City-County 
Public Library, Neel Parikh of the San Francisco Public Library, and Maria Salvadore 
of the District of Columbia Public Library. Institute participants carefully scrutinized 
the E-scale as one of the activities dufing the Institute itself and made dozens of 
valuable suggestions which will be incorporated into the scale as it is developed 
further. The E-scale was pretested for feasibility in two libraries in Wisconsin and six 
in California; information gained from the testing will also aid the development of the 
scale. At this time the E-scale has not been validated experimentally; while it may be 
useful for individual libraries, it is not appropriate for research use. 
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At an early stage of the E-scale's development it became dear that it would be 
most useful to actually create two scales, one for early childhood (ages 0 to 
approximately seven) and one for middle childhood (approximately ages eight to 
twelve). The developmental needs and the intellectual abilities of these age groups 
differ so greatly that they must be taken into account in environments for the two 
groups. The chosen point of distinction between the two corresponds to the division 
between two of Piagef s theoretical stages of child development, the preoperational 
stage and the concrete operations stage. 

Description of the E-scale 

The Environment Rating Scale for Public Library Children's Services consists of 
eight sections with varying numbers of individual items to be rated. The sections are: 
Safety and Comfort, Furnishings and Display for Children, Language and Literature 
Experiences for Children, Educational Experiences for Children, Physical Activities 
and Motor Skills, Creative and Cultural Activities for Children, Social Development, 
and Adult Use of Children's Area (staff and patrons). The scale also includes score 
sheets with space for comments, such as the feature that most affects each rating, and a 
graph to chart the scores in each section. All parts of the scale are very much in draft 
form at this writing. 

An appendix at the end of the E-scale lists each of the roles from Banning and Role 
Setting with the particular items that I associate with that role. I found that 
Preschoolers' Door to Learning and Independent Learning Center have the most items 
associated with them as the E-scale currently stands, while Research Center has few. 
Perhaps the provenance of the E-scale in an instrument designed for day care centers is 
partially responsible for the heavy emphasis on items appropriate to the Preschoolers' 
Door to Learning role. It may also be that the profession's recent interest in work with 
young children has influenced my emphasis on that role. Traditionally, librarians 
have viewed public libraries as the "people's university," a place where people could 
pursue individual interests outside of any formal education setting, and for this reason, 
the Independent Learning Center may have more items associated with it. As for the 
Research Center role, most librarians consider it appropriate for only the very largest 
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public libraries, and as I was most concerned with medium and small libraries, the 
E-scale as it stands may not pay enough attention to that role. 

How to Use the Environment Rating Scale for Public Library Children's Services 

The E-scale can be used in its entirety to evaluate the overall effectiveness of the 
children's area or individual items or specific sections may be rated after the planning 
and role setting process has resulted in the choice of roles for the library. In the latter 
case, one selects the items and sections that relate to the relevant roles with the aid of 
the appendix. I recommend that trie items in Safety and Comfort and Adult Use of the 
Children's Area be rated regardless of the roles selected as these items relate to all of 
the roles in Planning and Role Setting for Public Libraries. 

Each library staff will make fts own decision as to who should do the rating. In 
many situations it will be valuable to have two or more persons do the rating 
independently so that results can be compared and discussed. Examples of such pairings 
would be the library director and the children's librarian in a medium-sized library or 
the children's coordinator and the child. *n's librarian in a library that is part of a 
large system. Politically, it might also be useful to have a member of the reference or 
technical services staff rate the children's room along with a children's services 
staffer. It may be feasible and helpful to have a member of the public work with staff. 

In rating the library on each item, there are three methods for gathering 
information on which to base ratings: observation, conversation with staff and users, 
and documentation. Select a quiet time to look at the room for those items which relate 
to purely physical facilities. You may need a tape measure to assist you. To observe 
child and adult patrons, select a variety of times of the day and days of the week when 
various activities are going on. Be sure that you sample programs, after school hours, 
peak reference periods, and evenings and weekends if those are busy times at the 
library. As you look at what patrons and staff are doing and listen to their 
conversations, try to empty your mind of your preconceptions about what goes on in the 
children's area and look at it with the eyes of a newcomer. Remember that young 
children, especially, don't have a concept of "library." For them each space has somo 
potential; they will explore and manipulate whatever is there, regardless of adult 
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intentions about how space and objects are supposed to be used. For example, many 
toddlers love to spin paperback spinner racks for the fun of it, but the centrifugal force 
which pulls books off the racks creates a reshelving problem for staff and a potential 
hazard for the child. 

Some aspects of the scale require interviewing the staff to check such things as the 
frequency of changing displays and who does them, which outside agencies are 
contacted, how often contact is made, and what kinds of mutual activities take place 
with the agencies. As a practical matter, one must ask about the quality of climate 
control (if one does not usually work in the particular library) since it is rarely feasible 
for a rater to spend a year in a library to get an answer to questions about heating and 
ventilation! Tact, diplomacy, and an open, non-judgmental attitude are required to get 
accurate answers and to avoid making staff feel defensive around some issues. 

A third source of information that can be useful in rating the environment of the 
children's area is the documents that the library staff produce, such as annual or 
monthly reports and any results of consultation, output measures, or other evaluation 
studies. Usually these materials wiU not have been produced for the sake of the 
environment rating and they will be less likely to be biased because of that, though 
they may be biased for other reasons having to do with the purposes for which they 
were written. Documents serve to verify what has been observed or told to the rater(s) 
and may also add information difficult to obtain otherwise. For example, the number of 
contacts actually made with outside agencies. 

As the rater comes to a decision about each item, she indicates the score for that 
item on a Likert-type scale. The E-scale uses a seven-point scale with the odd numbers 
labeled Inadequate (1), Minimal (3), Good (5), and Excellent (7). The even numbers are 
intermediate points when the rating would fall between one of the odd numbers. If a 
library had all of the things needed for a rating of five on some particular item but only 
part of the things for a rating of seven, it would be given a six, for example. Most items 
on the scale carry some explanatory examples of what to look for; these are intended as 
examples only, not as requirements. Public libraries differ a great deal among 
themselves, and these local variations are desirable. One item on the scale asks the 
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rater to evaluate the opportunities children have to use language during library 
activities. The examples given are: finger plays and songs in stoyhours, book 
discussion, or creative writing groups. It might happen that the library under scrutiny 
did none of those things but did provide show-and-tell in preschool storyhours and 
storytelling contests or poetry writing fcr school age children. That library would 
receive a high rating on the "using language" item (especially if the activities were 
deliberately planned to give children opportunities for speaking and writing), even 
though staff did none of the things used as examples for that item 

Some items on the scale are discretionary as they may not be applicable to all 
librarie One of these is the item about a plac.3 to hang coats. In some parts of the 
county, coate are not often worn; so, this item is meaning^ss. Another such item is the 
one about library skills instruction. Depending on local availability of professional 
school media specialists, this item may also be meaningless, although public librarians 
should take care that what children are taught in school is transferable to the local 
public library situation. Catalogs and indexes often differ in significant ways, as do 
policies and procedures. 

Scoring and Interpreting Results 

When all items rated, the points for each section are added and located on the 
scoring graph By connecting the scores with a line, it is easy to see at a glance in which 
areas the library is strongest and weakest. Scores should not be averaged in an attempt 
to make a single index number. It isn't possible to construct a scale for this purpose that 
would have intervals all equally distant from one another; an average would be 
meaningless. 

When a partial rating is done, such as in the case of rating only for specific roles, it 
is possible to examine the scores either as a percentage of the possible total for the 
relevant items or to set a target level that individual items should reach. For instance, 
if the environment was being rated to see how well it suited the Formal Education 
Support Center role, the total possible points would be fifty-six (eight items each 
potentially worth seven points). The staff could decide ahead of time that frrty-eight 
points (85%) would be a reasonable indication that the library was succeeding. On the 
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other hand, one might decide that a rating of six or better on all items po taining to the 
Formal Education Support Center role was desirable; in which case one would pay 
particular attention to those items which fell below the desired level. Conversely, 
staff may decide that since the Community Activities Center is not a role that the 
library is trying to fulfill it is acceptable to achieve a Minimal rating on those items 
that do not overlap with the Formal Education Support Center and other roles that the 
library is working on. Decisions on the analysis and interpretation of results should be 
made before the rating is begun. 

Like any evaluation process, rating the environment is not a "one time thing." The 
first time one uses the E-scale one establishes a baseline for comparing oneself the next 
time, presumably after one has had a chance to work on the weaknesses found the first 
time through. If library staff decide to change the primary and secondary robs that 
the library will direct its efforts toward, a re-evaluation would be in order to rate *e 
children's environment for its effectiveness regarding the new roles. 

Examples of the Use of the Scale 

One of the first items on the scale is visibility of children's staff when patrons first 
enter the children's area. (See Figure 1). Two libraries near the University of 
Wisconsin— Madison achieved very different ratings on this item. The first library is a 
one year-old building in a small town. When you enter tne library, the children's room 
is easily identified because one can see some child-oriented displays through the large 
entryway. One has to pass through the adult section first, though, and there is no sign 
indicating that this is the children's area. The only staff desk is a multiple purpose 
circulation-reference desk to one's left as you enter. There is no desk for the children's 
librarian in the children's area proper, and the main desk is situated in such a way 
that staff cannot see into the children's room from the main desk. Unless a patron 
knows the library staff, he or she will not know who has knowledge or responsibility 
for helping children or adults with the materials in the children's room. This library 
received a rating of 1 (Inadequate) on this item. 



UNIVERSITY OF WISCONSIN— MADISON 



WUlett 



Figure 1 



Item Inadequate Minimal Good Excellent 

1 2 3 4 5 6 7 

1. 

Staff visibility No staff visible at Staff are visible Staff smile and give In addition to 

when patrons any time when patrons but do not approach verbal welcome to friendly welcome 

ei ter children's enter. children or adults, patrons. staff convey 

room. readiness 

to assist patrons. 



A second library received a rating of 7 (Excellent) on the same item because the 
children's reference desk is the first thing o ne sees. On entering the children's room 
through wide double doors, the desk, staffed at all times, faces incoming patrons. 
Because staff nearly always look up at patrons and ask if they need assistance, the 
Excellent rating was deemed appropriate; if staff were not welcoming, the rating would 
have fallen to 6. 

Of course, distinctions are not always so obvious. Two libraries provided hands-on 
materials to reinforce perceptual and fine motor skills. (See Figure 2.) The items 
included puzzles, toys, and materials for drawing. One library was rated 3 (Minimal) 
for this item because there were only a few such items and they were placed out of 
reach for the preschoolers who were their intended users. The second library provided 
more items, placed them on tables where children could easily reach them, and 
displayed some of the drawings children made. The rating was 5 (Good) on the 
perceptual and fine motor skills item. If there had been any indication that staff 
interacted with children working with materials and that the presence of the 
materials was related to the library's goals and objectives rather than just that it was 
"nice" to have such things, the rating might have gone as high as 7. 
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Figure 2 



Item 



Inadequate 

1 



2 



Minimal 

3 



4 



Good 

5 



6 



Excellent 

7 




Large variety of 
itema for all ages 
served. Regular 
budget includes 
purchase of 



for fine motor skills. 



toys, craft 
materials. 



Conclusion 

Before the Environment Rating Scale for Public Library Children's Services can be 
published, further revision and field testing will be necessary to validate the scale. 
While there is much work to be done on it, the foregoing should give the reader a taste 
of what the E-scale is intended to do. Readers who would like a copy of the current 
draft of the scale should write the author; comments and suggestions for areas that 
should be included in the E-scale are welcomed. 

Whether one uses the E-scale, another device, or just one's "eyeballs" to evaluate 
the public library's environment for children, there are three concepts to keep in mind. 
The first is to evaluate from the general position that the overarching mission of 
children's services is to tnhance child development, not only for the sake of the adult 
the child will become but also for the sake of the child in the here-and-now. The 
second is to evaluate in terms of the specific roles selected for the individual library. 
The question here is, How well does the environment allow us to play our chosen 
roles?" The third concept is that evaluating the environment includes looking at more 
than the physical facilities. Staff behaviors and attitudes are part of the environment 
as are the activities and materials provided for children. The Environment Rating 
Scale for Public Library Children's Services attempts to keep all three concepts in view. 
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Introduction 

The previous chapter presented a variety of ways of gathering evaluation data 
from interviews and questionnaires. Sometimes evaluation data are obtained as a 
natural part of doing business or through limited small-scale investigations that 
involve counting of predefined items. This chapter presents several examples of and 
perspectives on this everyday font of evaluation. 

Vie have already presented two selections on the topic of sampling. While it may 
be generally understood that sampling is required in order to carry out a survey of 
community r cedents, libraries too often continue to use total counts when tallying such 
indicators of service as reference transactions. Output Measures for Public Libraries, 
2nd edition uses sampling in order to increase the accuracy of data for many of its 
measures. We have reprinted the section of the manual that deals with sampling 
because it illustrates how the principles of sampling apply specifically to measures of 
library services. 

Mary Jo Lynch, Director of the Office for Research of the American libranj 
Association, has led efforts to standardize the collection of library statistics and to 
speed the appearance of reports so that they can be used for decision-making. She 
speaks to the importance of providing data on children's services especially and to 
current activities that can have important implications for those concerned with 
public library services for children. 
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In 1985, a Library Services and Construction Act grant allowed the adaptation of 
Output Measures for Public libraries to measure services for children. The selection 
included in this Sourcebook gives the rat 'omle for the project, a sense of what was 
accomplished, clarification of a number uj issues that arise when measuring services 
for children, and procedures and results for two of the measures used. 

Finally, Craighton Hippenhammer's article discusses how the output measures 
concept can be applied to management of children's library collections. 
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Sampling 



Nancy Van House/ etal 

Excerpted by permission from 
Output Measures for Public Libraries; 
A Mar-tal of Standardized Procedures, Second edition, 
Chicago: ALA, 1987. 



Sampling is the process of selecting a subset of a population (of people, objects, or 
events) from which to draw conclusions about the larger group. The key to sampling is 
that the group studied must be representative of the population to which the results 
are to be generalized. The guidelines presented here and in Chapter 5 are all aimed at 
ensuring representativeriess. 

Before selecting a sample, the population to be studied should be carefully denned. 
This is to ensure that the sample represents the larger population in which the library 
is interested. For example, if children and non-English-speaking adults are excluded 
from the Materials Availability Survey, then the results for the Materials 
Availability Measures do not apply to all library users, but only to English-speaking 
adult library users. 

To avoid unintended biases introduced by the people drawing the sample, specific 
criteria are established, and every person, item, or transaction meeting those criteria is 
selected. This is why it is important to follow the instructions in this manual for the 
measures that use samples. For example, if the reference staff skip recording reference 
tMMMdioiw when they are very busy, the Reference Completion Rate will only 
represent the level of service when the desk isn't very busy. But the Reference 
Completion Rate may be higher when the staff are less hurried, lower when they are 
busy. 
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Sample Size 

The immediate practical problem that arises when sampling is: How large a 
sample is needed? The larger the sample, the greater the level of effort, A library will 
want to use samples as large as necessary, but no larger. 

Samples result in estimates. The larger the sample, the more precise is the 
estimate, that is, the closer the sample result is to the true value (the population 
parameter, or the value that would have been derived from collecting data on the 
entire population instead of just a sample). The sample size needed depends on how 
accurate an estimate is needed. 

The measures that use sample data fall into two categories: 

• Those that estimate the number of activities or items: 

Library Visits per Op^t 
In-Library Materials Use ^er Capita 
Reference Transactions per C.*pite 

• Those estimating proportions or percentages: 

Title Fill Rate 
Subject and Author Fill Rate 
Browsers' Fill Rate 
Reference Completion Rate 

(Document Delivery falls somewhere in between; the nvitructions in 
Chapter 5 explain how to determine its sample size.) 

For the first group of measures, those estimating the number of activities or items, 
sample sizes have been developed and are described in Chapter 5. Generally, 
experience has shown that for most of these measures an adequate sample size is one 
week's worth of transactions. For Library Visits per Capita, for example, it is 
recommended that the library count visits for one week and multiply by 52 to estimate 
ANNUAL NUMBER OF LIBRARY VISITS. 
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For the second group of measures, those that reflect proportions, libraries can choose 
their sample sizes based on the precision of estimate that they need. For those that 
prefer not to choose their own sample size, the recommended sample size is 400 
transactions, for example, 400 each of TITLES SOUGHT, SUBJECTS AND AUTHORS 
SOUGHT, BROWSERS, or NUMBER OF REFERENCE TRANSACTIONS— whatever 
the appropriate transactions are for the measured to be used. For libraries that find 
400 transactions excessive, smaller samples are acceptable, but no smaller than 100 
transactions of each kind. 

Using samples smaller than 400 results in less precise estimates. A less precise 
estimate may be sufficient, however. The choice of sample size should be based on the 
library's needs and the level of effort to be devoted to data collection. The next section 
explains what happens to the precision of the estimate with different sample sizes. 

Confidence Intervals 

Regardless of whether the libiary chooses its own sample size or uses the 
recommended size, to interpret the data you need to know how far off the estimate 
might be from the true value, that is, from the figure you would have obtained by 
collecting data for the entire population. The data coordinate will do the actual 
calculations (using Figure 1), but library management and the planning committee 
(where there is one) will find the following discussion helpful in understanding the 
results. 

The possible difference between the estimate derived from a sample and the true 
value for the entire population is expressed using a confidence interval. The confidence 
interval consists of the value for the sample plus or minus the possible "play" or 
margin, that is, plus or minus the maximum possible distance that the estimate could be 
from the "true" value. The result is a range within which the "true" value falls. 
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Figure 1 

Margin of the Fill Rate Estimate 
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Note: Based on a 2-tailed test, 80% confidence level 

Example; A library with a sample of 300 and a Title Fill Rate of 60% has a 
margin of 4%. The confidence interval for its Title Fill Rate is 56% to 64%. 



Confidence intervals are important because a simple Fill Hate may be misleading. 
For example, in Figure 2, Branch Cs Title Fill Rate is 70 percent, so it appears that 
Branch C is not doing as well as Branch A. But actually the branch Title Fill Rates may 
be the same, because Branch A'sTitle Fill Rate confidence interval goes as low as 70 
percent, and Branch Cs goes as high as 75 percent. So they could actually be the same, 
somewhere in the range of 70 to 75 percent. The same would apply to a library that got 
a Title Fill Rate of 75 percent, plus or minus 5 percent, one year, and 70 percent, plus or 
minus 5 percent, the next. Service levels may not actually have changed from one year 
to the next. 
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The size of the margin depends on the size of the sample and the results obtained. 
The smaller the sample, the greater is the possible margin in the estimate. The next 
section tells you how to figure out what your margin is. The point is that the sample 
size only need be large enough to create a confidence interval that meets the library's 
needs: the less precision the library needs, the smaller the sample that will suffice. 



Figure 2. 
Comparing Confidence Intervals 
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.J |- = CONFIDENCE INTERVAL (±5%) 



The methods that follow are appropriate for Title Fill Rate, Subject and Author 
Fill Rate, Browsers' Fill Rate, and Reference Completion Rate. For simplicity, the term 
Fill Rate is used in the discussion below. 
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The first section below is for libraries that want the simplest approach. At a 
higher level of effort, the library can determine the precision of its estimate, that is, 
the width of its confidence interval, by adjusting its sample size. This is covered in the 
second section below, Calculating Sample Sizes. 

Using the Suggested Sample Size 

For the Fill Rates, this manual suggests a sample size of 400 transactions for each 
of the measures. If 400 transactions are more than the library can feasibly handle, the 
sample can be smaller, butno smaller than 100 transactions. That is, for Title Fill Rate, 
we recommend between 100 and 400 TITLES SOUGHT; for Subject and Author Fill Rate, 
between 100 and 400 SUBJECTS AND AUTHORS SOUGHT; between 100 and 400 
BROWSERS for Browsers' Fill Rate; and between 100 and 400 REFERENCE 
TRANSACTIONS for Reference Completion Rate. These should be collected over a 
period of no more than two weeks. (For more on this, see the instructions for the 
Materials Availability Survey in Chapter 5.) 

Remember that these sample sizes refer to usable transactions. One Materials 
Availability Survey form, for example, may contain more than one TITLES SOUGHT 
and SUBJECTS AND AUTHORS SOUGHT, and may or may not represent a BROWSER. 

After collecting the data and calculating the Fill Rate(s) following the instructions 
in Chapter 5 of this manual, use Figure 1 to figure the confidence interval for each 
measure.* 

To use Figure 1: 

1 . Find the sample size closest to yours across the top. Your sample size is 
the total number of transactions of this kind in your sample (e.g., 
TITLES SOUGHT, SUBJECTS AND AUTHORS SOUGHT, 
BROWSERS, or REFERENCE TRANSACTIONS). Your goal may have 
been 400 (or 100), but you may have ended up with more or fewer. 



* Figure 1 is at an 80% level of confidence. This is lower than is customary for research, but 
probably adequate for management decision making. At this level of confidence, we can 
say that, in the long run, 80% of the confidence intervals obtained using this method will 
include the true (population) parameter. 
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Down the left size of the table, find the Fiu Rate closest to yours. 



3. The figure at the intersection of these two is your margin. 

4. Your Fill Rate interval is between the figure that you calculated from 
your survey results PLUS your margin, and that same figure MINUS 
your margin. 

Example: A library's users report 200 TITLES SOUGHT. Sixty percent 
were filled. Using Figure 6, the margin for 200 at 60 percent is 4 percent. 
The true Title Fill Rate for this library is 60 percent plus or minus 4 
percent, or between 56 and 64 percent. The confidence interval for this 
library's Title Fill Rate is 56 to 64 percent. 

Calculating Sample Sizes 

Calculating your own sample size is slightly more complex than using the suggested 
sample size. It is most appropriate for libraries that have used a measure before and 
want their estimate to fall within a specified range. 



1. Estimate what the Fill Rate is likely to be (from previous years' data, 
from neighboring libraries, or from an informed guess). 

2. Decide what size margin you need and find it in Figure 1 in the row 
with the expected Fill Rate. 

a . The size margin you need depends on how precise an estimate you 
need: is plus or minus 5 percentage points acceptable, or do you need 
plus or minus 3 percentage points? 

3. The sample size at the top of that column is the target. (Remember 
that these are usable transactions; see the instructions for the 
Materials Availability Survey in Chapter 5 to translate this to MAS 
questionnaires.) 

4. After the date have been collected and analyzed, it is likely that the 
final Fill Rate and/or sample size is not quite what you expected. Use 
the method under Using the Suggested Sample Size, above, to calculate 
the margin from the actual data. 



Example: The same library decides the next year to determine its own 
sample size. It expects its fill rate to go down to about 55 percent because 
its new materials budget was cut It wants to bring its margin down to 
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plus or minus 3 percent. Using Figure 1, it needs 400 TITLES SOUGHT 
(between 50 and 60 percent fill rate, 3 percent margin). 



A more precise estimate requires a progressively larger sample size. There is a 
direct trade-off between level of effort and precision. 

Multiple Outlet Sample Sizes 

In a library with branches, a decision must be made about the results needed: 

1 . One figure for each measure for the entire library taken together, or 

2. A figure for each branch individually as well as the library as a whole. 

Once this decision has been made, the data coordinator uses the discussion that 
follows to determine sample sizes. 

For the measures for which the library counts transactions during the week, each 
outlet simply collects data for a week; how those data are combined to a system-level 
result is explained below under Data Analysis. For the Fill Rates, this decision affects 
the total sample size and how the sample is distributed among the branches. 

Branch-level Results: Using the recommended sample sizes, each outlet should 
have a minimum of 100 transactions (400 is better). If you calculate your own sample 
size, use the method described above under Calculating Sample Sizes, repeating for 
each branch individually. To calculate the overall library-level measure from the 
branch samples, see, under Data Analysis below, Outlet and Library-level Data. 

Overall Library-level Only: The sample size required for the library as a whole is 
either the recommended sample size (preferably 400 but at least 100), or is calculated 
using the methods described under Calculating Sample Sizes. Then the sample is 
distributed among the branches according to their relative shares of total system 
activity [OMPL2 has a form on page 22 that may assist this calculation.]: 
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1. Use the previous year's circulation data to determine the percentage of 
system activity contributed by each branch. 

Example: Total ANNUAL CIRCULATION for the system was 100,000. 
The Main Library's circulation was 75,000, or 75 percent. Branch A s 
circulation was 25,000, or 25 percent. 

2 Determine the sample size needed using either the recommended 
sample or the method above for calculating your own. 

3 Multiply the sample size by the percent of circulation for each branch 
to determine how many transactions should come from each branch. 
This is the target sample size for each branch. 

Examplr. The library needs a sample of 400 to ^determine its /Title Fill 
Rate. 300 TITLES SOUGHT (75 percent) should come from the Main 
Library, 100 from Branch A. 

Simply disputing the sample equally among the branches, without correcting for 
activity level, would give smaller branches a disproportionate influence on overall 
results. Smaller branches cor tribute less to the total volume of system service and so 
should count proportionate!} - less. 
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National Statistics on Public Library Service to Children; 
Why and How to Get Them 



Mary Jo Lynch 



So far, the institute has focused on measurement at the local library level. I know 
the topic of this institute is evaluation, which is different from measurement, but I also 
know that measurement usually comes first. You have to measure something-find out 
how long or high or heavy it is-before you can evaluate how good it will be for the 
purpose you have in mind. Since Joey Rodger .will not be here, lU recite for you the 
evaluator-s cheer which I first heard from her: "2-4-6-8, measure; then evaluate!" 

For many years I have been working to improve the measurement of all aspects of 
library service on the national level. In connection with tho*2 efforts, I've been wanting 
to talk to a group like this for a long time about national statistics on public library 
service to children. What national statistics you might ask? There aren't any, are 
there? You are right. There aren't. Thaf s the problem I want to explore hoping that, 
together, we can seek a solution. 

That topic may seem a little off target at this institute, but I believe it fits and 
adds an important dimension. Collecting data for regular reporting and for special 
evaluative studies at the local level is essential for effective management of children's 
service. But it isn't enough if we want to see public library service in general and service 
to children in particular recognized as an important part of American life. National 
statistics are needed to make that possible. 

II. Background 

Two things happened ft'oout five years ago to convince me that this was an issue 
that needed attention. The first was a call I received from Linda Silver who was 
chairing an ALSC committee charged with investigating what national statistics were 
available about public library children's services. I studied the statistical reports I 
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usually consult when I'm asked a questions about public library statistics and came up 
with nothing. The National Center for Education Statistics (NCES) didn't have 
anything in the sample survey reports they did every four years. The Allen County 
Public Library didn't have anything in the biennial data they published from public 
libraries serving populations of over 100,000, nor did the Urban Libraries Council in the 
biennial data they gathered from their members. We did find out about a few 
state-level reports and were able to locate, with the help of Herbert Goldhor at the 
University of Illinois, some large public libraries that included statistics on children's 
services as part of their regular annual reports. But there was nothing that would 
enable us to say: here are figures that describe public library service to children on the 
national level. 

At about the same time, I read a book by economist Lawrence J. White entitled, 
The Public Library in the 1980's: The Problems of Choice. True to his training as an 
economist, White viewed the public library from a public policy perspective. He 
questioned the legitimacy of the public library's claim upon public funds and concluded 
that, although the public library was what economists call a "public good," and thus 
entitled to public monies, that claim was not strong enough to warrant full support from 
that source. He recommended, therefore, that p« s libraries begin to charge fees for 
many services. 

I have problems with many of White's conclusions and recommendations, but am in 
complete agreement with what he has to say about public library service to children 
and about the need for national statistics about that service. 

Chapter 9 of White's book is headed "The Special Problem of Service to Children 
and Students." The Chapter begins with the following quote from Margaret Edwards: 

Public libraries consider young people an interruption to business, yet 
they are the business, more so than the middle-aged adults the 
librarians are waiting for. (p. 97) 
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In an earlier chapter, White had presented statistics that showed public library 
circulation to children and students to be over half of the total. (The data were 
incomplete and not up to date, but White believed they supported his conclusion.) Yet 
most of the surveys of public library use focus on service to adults and service to adults is 
what public library directors emphasize. Those facts are very old news to this group. 
But Larry White puts them in a public policy perspective when he says: 

The implication of these criticisms is important to an assessment of the 
social value of the public library. Service to children and students 
comes closest to satisfying that library's goals of encouraging literacy 
and furthering education; as I will argue in chapter 11, the case for 
public provision of free library services is at its strongest when applied 
to children's and student's service. If the library's main focus is 
elsewhere, and if it is short-changing service to child and students, 
then the overall case for the free public library as it is presently 
constituted becomes weaker still, (p. 99) 

The implications of White's comment are clear-public library advocates are 
missing the boat when they ignore or minimize service to children. White elaborates on 
this point in a later chapter when he explains the economic concept of "externalities." 
The point here is that although it may seem that in providing free public library 
service, government is favoring one group of citizens unfairly-the group who use 
Ubraries-there are benefits that flow to the whole community because of the service to 
a few. White explains that: 

In this context, then, the case for public library service to children and 
students appears to be very strong. Service to children encourages them 
to be more literate and to develop a greater appreciation for books and 
for the printed word. Public library services to students are an extension 
of their formal education. Making library services available at zero 
cost to these users encourages their use of the library and could be 
justified in the same way that free public education to children is 
justified, (p. 131) 

White goes on to show that the case for external or spillover benefits to others in 
society is much less strong when applied to adult services. I have some problems with 
his analysis here, but fully accept his earlier conclusion— that service to children is 
very important from the public policy perspective. 



UNIVERSITY OF WISCONSIN-MADISON Lynch 



195 



ERIC 197 



White's final conclusions and recommendations bring joy to the heart of one who 
cares about library statistics. His first recommendation is that the federal government 
should collect comprehensive statistics annually from public libraries and his second 
recommendation is that we must have statistics about the use of public libraries by 
children and youth. White criticizes that National Center for Education Statistics for 
a haphazard approach to public library statistics, but he puts the primary blame on 
librarians. White says: 

This haphazard approach to statistics is not accidental. It is 
indicative of a profession that has not understood the value of 
statistics and, indeed, is fearful of their use. Both the practice and the 
attitude must change. To paraphrase the quotation from Lord Kelvin 
cited at the beginning of Chapter 3, if you haven't measured something, 
it's awfully hard to talk sensibly about it. (p. 161) 

Let me give you the full quotation from Lord Kelvin: 

When you can measure what you are speaking about and express it in 
numbers you know something about it; but when you cannot measure it, 
when you cannot express it in numbers, your knowledge is of a meager 
and unsatisfactory kind. (p. 15) 

That's the situation we have been in for some time regarding public libraries in 
general and service to children in particular. Things are definitely looking up for 
general public library statistics (some exemplary projects are outlined in Section HI). 
Statistics on service to children however, are not doing as well. I hope this group can do 
something about that. 

III. The Federal-State Cooperative System 

I want to highlight a series of projects I began working on shortly after my encounter 
with Larry White. The first project was entitled "Analysis of Library Data Collection 
and Development of Plans for the Future." That project involved examination of the 
statistics collected by NCES from various types of libraries and the progranu of any 
other agencies collecting library statistics especially at the state level. The document 
included recommendations for future action by NCES and prepared model questionnaires 
for NCES to use with each types of library. A summary of the final report on the project 
appeared in the Bowker Annual for 1985. 
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A key component of the project was a detailed analysis of the forms used by states 
to collect data from libraries. The three-volume appendix to the report contained 
charts that show every item collected by any state and all states tnat collect the item. 
Two of the three volumes report on public library statistics— a total of 377 pages. 
Clearly there was a lot of variety in state data collection from public libraries in 1983- 
84! We found that all fifty state library agencies collected statistics annually from 
public libraries in their states. The general topics they collected data on were similar, 
but the specific items were not and therefore the results were not comparable from state 
to state. The report recommended that NCES persuade the states to collect a limited set 
of key items in a standard way and report them to NCES so that national summaries 
could be created. Before that report was submitted, the idea was presented to the Chief 
Officers of State Library Agencies (COSLA) which supported it in principle. 

Did the states collect statistics about servce to children? The answer some did, but 
not much. My article about this in the Winter, 1985 issue of Top of the News reported 
that: 

Twenty-four states included an item on circulation of materials to 
youth, but the language varied. Some states asked about "juvenile 
circulation"; some included a question under the heading "Circulation" 
for "Books— juvenile" or for "Fiction— juvenile" and 
"Non-fiction— juvenile"; some asked about circulation of "Printed 
materials— juvenile".... Twelve states asked about number of programs 
and/or number of attendees for "juvenile programs." 

The March 1984 fi- . report recommended that at least one item be collected 
nationally from public libraries— circulation of juvenile materials. 

Shortly after the November 1984 report was filed, ALA sent a proposal to the 
Department of Education asking for funds to conduct a pilot project that would work 
with a small group of states (five to seven was suggested) to explore the feasibility of a 
system ihat would coordinate the annual collection of data from public libraries in the 
fifty states with the periodic reporting of national statistics on public libraries by 
NCES. 
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Two units of the Department of Education provided financial support for the 
project— Library Programs and NCES. In October 1985 the project began and a letter 
went out immediately to all fifty chief officers of the state library agencies inviting 
them to participate in the pilot. Twenty states volunteered. When the Advisory 
Committee met in November 1985 they decided to accept everyone who volunteered and 
to rearrange the project to allow it. By the time the project ended, there were fifteen 
official participants: California, Colorado, Florida, Idaho, Indiana, Minnesota, New 
Hampshire, Ohio, Oklahoma, Oregon, Pennsylvania, South Carolina, Utah, 
Washington, and Wyoming. 

States were free to collect whatever they wanted as long as they collected a certain 
set of items using standard definitions and sent data on those items to NCES for 
incorporation into a national report. States could continue collecting data at whatever 
time of the year was best for them. This would mean that any aggregate report would 
contain data collected at different times but sent to NCES at a standard time. The 
Advisory Committee spent a good bit of time at their first meeting revising the list of 
items and definitions that had been recommended in the report of the previous project. 
Those items and definitions were to be revised two more times before the pilot project 
was finished! The question about circulation of children's materials stayed on the list 
through the first two revisions but was finally dropped because many participating 
states complain* 1 that most libraries could not supply the figures. 

The pilot project took two years to complete. During that time there was a 
workshop for participating states and a lot of communication and coordination 
regarding a key component of the project— data were to be sent from the states to NCES 
in machine-readable form. 

At the time of the March 1986 workshop only four states knew they could send data 
to CES in machine-readable form — two on magnetic tape produced from a mainframe 
and two on floppies from a microcomputer. By the time the project was finished all were 
able to do so, albeit with different levels of skill and ease. Twelve of the fifteen pilot 
states sent 1986 data to NCES by the time the pilot project ended and the other three 
did so soon after that. 
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The pilot project was officially completed at the end of August 1W w*an a fc*l 
report was submitted. That report recommended that work begin immediately » 
expand the pilot project into a 50-«tate system. By that time a library statistics 
coordinator had been appointed at NCES and had begun to put the data from pUot 
project states onto the NCES mainframe and analyze it. Both the final report on the 
pilot and that appointment coincided with a renewed interest in library statistics at 
NCES, due to the discussion of HR5 which eventually became Public Uw 100-297 (the 
Hawkins-Stafford Elementary and Secondary School Improvement Act). This law gave 
NCES greater authority and responsibility, and specifically charged NCES to collect 
statistics about libraries. The need for a federal-state cooperative system for public 
library data is specifically mentioned in the law. 

In February 1988, a "Memorandum of Understanding" was signed between NCES and 
the National Commission on Libraries and Information Science (NCUS) whereby 
NCUS would coordinate the work of a Task Force established to plan the expansion of 
the pilot project into a 50-state system. That Task Force met monthly from March 
through September 1988 and developed a plan for the Federal-State Cooperative 
System for Public Library Data Collection (FSCS). One key component of the plan is 
that the pilot project's list of eighty-one items was reduced to forty-one. The other 
basic ideas remain the same, however. 

Implementation of the plan began immediately with a workshop in Annapolis on 
December 5-8 for state data coordinators. Forty-eight states and the District of 
Columbia sent a representative to the Annapolis meeting where they learned what 
would be required of participants and what help would be available from NCES and 
NCLIS. 

As of May 1, 1989, forty states were expected to submit their most recent data to 
NCES in July 1989. There will undoubtedly be some problems with the forty and there is 
much work to do to increase forty to fifty. But it seems very likely that FSCS will exist 
very soon. Plans are also underway to change the agency linking states to NCES from 
NCUS, where that work is done now, to COSLA, the Chief Officers of the State 
Library Agencies. 
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2 There a a minor daspute about the vahdity of using circulation of children's 
material* as a measure of service to children once children take out other 
materials and those who are not children take out materials designed for 
children. 

3. There is a major dispute about the age at which a person stops bong a child and 
starts being a young adult 

Leaders in the community of children's library service need to reach a reasonable 
consensus on those issues first and then convince other practitioners to collect data. Then 
you can approach state library officials about incorporating data items into state forms 
and eventually into national reporting. It can't happen tomorrow or even next year, but 
it can happen. 
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Two models come to mind as 1 think about how to do this. One is what 1 call the 
California model. Ned Parihh of the San Francisco Public Library, representing the 
Children's Services Management Consortium, asked the California State Library to 
collect more statistics about childrm's services on their annual survey. Before doing so, 
the Consortium polled their member libraries about what data they could supply 
easily. Armed with that evidence, the Consortium convinced State Librarian Gary 
Strong that questions should be added to the California form about prograinming for 
children and expenditure for children's materials in addition to a question on 
circulation. 

A second model 1 would like to offer comes from the reference service area. Many 
years ago, the Committee on Reference Statistics of the Library Adrninistration and 
Management Association's Statistics Second became convinced that a count of reference 
questions ought to be part of the periodic surveys that NCES conducted of public 
libraries and academic libraries. So they worked out definitions that separated 
substantive "reference questions" from "directional questions" (i.e., Where is the 
bathroom, the photocopier, the pencil sharpener). It took a while to get the language 
right and to convince reference librarians to collect the data. But the committee kept at 
It and was successful in getting their definitions used in state and national statistical 
surveys. We are not out of the woods yet on that one as national sfctistics on reference 
service in public libraries are incomplete at this time. But questions on reference are in 
FSCS. So the situation should improve soon. The same cannov be said of national 
statistics on children's service— yet. 

You may wonder why I have talked at great length about FSCS-the Federal State 
Cooperative System-Hind said nothing about PLDS-PLA's Public Library Data 
Service. PLDS and FSCS are very similar in that they both collect basic data annually 
from public libraries-data that describe staff and collection and budget and ser/vices. 
They are also very different in several ways. Two differences assume special 
importance in the present context. 

1 . When FSCS is complete in a couple of years, it will report on all public 

libraries in the U.S. PLDS focuses on public libraries serving populations of over 
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100,000. Since less than five percent of the public libraries in the U.S. serve 
populations of that size, PLDS is of less importance from a broad public policy 
perspective. The libraries in that five percent do serve the largest population 
centers, but public policy is concerned with everyone. 

2. PLDS was started by a PLA committee and is now i d vised by a PLA committee. 
The process of introducing an item on children's service into the PLDS 
questionnaire is much less complex than introducing a new item into FSCS 
which has been and will be shaped by the fifty states. 

My advice on how to do it is similar to my advice on how to get an item into FSCS. 
It should be easier to get into PLDS, however, because there is less bureaucracy involved 
and because there are fewer libraries involved. 

Why am I so concerned about this? Because I believe with Larry White that from a 
public policy perspective service to children is an essential part of public library 
service. However, it is currently invisible at the national level. When people call my 
office for statistics about public library service to children, I can't give them much. And, 
I am haunted by a quip some professor made one day in a political science class: 

"If you can't count it, it doesn't count." 

That offends my humanistic soul, but I'm afraid if s true when you're dealing with 
politicians and economists and journalists. Since we know children's services do count, 
we need to make the numbers available in order to convince others. Of course 
measurement isn't evaluation but it is something. 

I'll risk your not already having heard Peter Drucker's argument for using 
quantitative surrogates for qualitative achievements. The example Drucker uses to 
illustrate his point is one that could easily be adapted to the library situation. Drucker 
notes that a minister cannot measure the number of souls saved, but it is possible to count 
how many people come to service regularly. Librarians who dismiss the importance of 
statistics because they don't tell everything about services to children are missing an 
opportunity to use statistics creatively as surrogates for more important things. 
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V. National Statistics Now 

Before I close let me mention two shining lights in the darkness surrounding public 
library children's services. One is a statistic that has been published annually for 
years. The second is a national survey that was conducted this winter. 

The annual statistic was developed many years ago by Herbert Goldhor of the 
University of Illinois. Goldhor used techniques designed by economists to create the 
"Index of public library circulation" and the "Index of public library expenditure " Both 
are reported for 1986 in the American Libraries article that was photocopied for 
distribution at this meeting. Notice every year is expressed in comparison to 1980 
which is expressed as 100. 

Since circulation is separated into adult and children this index gives us real 
evidence of the growth of circulation of children's materials. If s the first piece of 
evidence I quote when someone calls wanting to know whether children are using 
libraries less today because of television or because of illiteracy. I too find the index 
surprising in view of those factors but I checked the numbers with Susan Roman and she 
tells me they correspond with what she hears from the field. The Illinois index is not 
widely known but it should be. Were local libraries to keep statistics on circulation of 
children's materials, those figures compared to this national index would iUumine how 
a local library compares to the national average. 

The second light in the darkness promises to be even brighter. The Fast Response 
Survey System of NCES (FRSS), established in 1975, is designed to collect small 
amounts of issue-oriented data for educational planning and policy. It does this quickly 
and with minimum burden on respondents. FRSS is available to policy analysts, 
planners, and decision makers in the Department of Education. 

Ray Fry of the Library Programs unit lias been successful in persuading NCES to do a 
fast response survey on public library service to young adults in 1987 and results were 
published in 1988. In the Spring, 1989 issue of Journal of Youth Services in libraries an 
article by Mary K. Chelton describes the questionnaire design process and comments on 
the results. 
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When several of us learned about the Young Adult survey, we decided to ask for a 
similar survey on public library service to children. I wrote to NCES, after conferring 
with ALSC, PLA, and NCLIS. NCES agreed and Ray Fry ag? in acted as a link between 
NCES and the library community. Susan Roman and I served on the committee that 
advised on the survey. Also serving were Mary Somerville, Maggie Kimmel. Barbara 
Immroth, and Jane Botham. We worked very hard but we didn't get everything we 
wanted. The length of the questionnaire is constrained by FRSS procedures and some of 
our suggestions on wording could not be implemented. In general, we are pleased and 
look forward to tne results which should be available before the end of 1989. This will 
be the first national data ever on public library service to children. It will be great to 
have now in the Year of the Young Reader as libraries join the First Lady in a campaign 
for family literacy. The survey data will furnish a base to build on in the future. I look 
forward to working with you to do that. 

VI. California Raisins 

You are the first librarians outside of ALA headquarters to hear about some other 
data that will be available this fall related to children and literacy. Several months 
ago, I got a call from someone at a New York public relations firm wanting statistics on 
public and school libraries — how many there were, did children use them, was 
circulation up or down. I was very busy that day but the caller was pleasant and 
persistent and told me she was working on the California Raisins account. I was amused 
by the dancing raisins so I spent more time than I usually do with commercial callers. 
After doing what I could, I referred her to AASL for more detail on schools, and thought 
no more about it. Several weeks ago I got a letter on California Raisins Advisory Board 
letterhead asking if ALA would cooperate with them on a project that involved: 

1. interviews with kids regarding attitudes toward reading; 

2. enrolling kids in a "raisin reading" club and giving them prizes when they had 
read a certain number of books. 

The California Raisins Advisory Boaid wanted to do something to promote literacy 
and needed ALA's help on both components. I turned the matter over to PIO [Public 
Information Office], ALSC, AASL, and OLOS [Office for Library Outreach Services] 
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and things are moving frst: PIO is working with them on materials and arranging an 
appearance of the dancing raisins at the Annual Conference in Dallas. 

The interviews will be done this summer by a Chicago firm that specializes in 
interviews with children. They will do "mall intercept" interviews with fifty to sixty 
children between the ages of 6 and 11 in each of ten large cities. Children visiting mall! 
with parents will be asked a series of attitvdinal questions. I don't have the draft 
questions yet but once we get them I could really use some advice from people attending 
this institute who have expertise in interviewing children. This is a wonderful 
opportunity to influence research being done on someone else's nickel. 
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Project Description 

This report is the result of a pilot study of the use of output measures for measuring 
children's services in Wisconsin public libraries which took place in March-April 
1985. 

This project is part of a long-term effort on the part of Wisconsin children's 
librarians to improve the measurement and planning of services. In 1979, children's 
librarians conducted a two-day review and discussion of services. In 1981, a statewide 
survey of children's librarians identified the need for improved administrative and 
management skills. In February, 1984, a group of 24 Wisconsin children's and young adult 
librarians met a b ain to discuss services. This group was especially concerned about 
gathering data which would be useful in documenting efforts to improve services. 
Members had heard about output measures, and they wanted to know more about them 
so they instructed the state outreach consultant to write an LSCA (Library Services and 
Construction Act) grant application which would provide them with a continuing 
education experience to learn about output measures and also the opportunity to test the 
measures with children. The grant was written, awarded, and implemented during FY 
1985. 

The staff consisted of Gloria A. Waity, Cooperative Children's Book Center 
(CCBC) Outreach Consultant and Project Administrator; Joan A. Braune, Project 
Coordinator; and Douglas L Zweizig, Project Consultant. Waity wrote the grant 
application and was responsible for administration and coordination with the 
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children's librarians. Braune was responsible for the organization of the meetings, 
preparation of project materials, and project communications. Zweizig provided 
training and consultation at the three meetings and is the author of the draft manual 
that: was tested and of this report. The measures themselves were conducted by the 
librarians listed in the Acknowledgements. They represent libraries from almost every 
one of the 17 regional library systems in Wisconsin. The project staff members functioned 
as organizers and educators, but the project itself belonged to the librarians. The staff 
consulted with them, but group decisions as to what measures would be implemented, 
what definitions would be used, and other such decisions were made by the librarians. 
Because of the group decision-making and because this was not a project or an activity 
imposed upon them by an outside authority, a strong sense of ownership of the adapted 
measures and the statistics which composed the data occurred. 

Decisions on what to measure and how to measure it were made during the group 
meetings. Three were scheduled as part of the project, but any group wishing to 
replicate the experience would be wise to plan four sessions. At the first meeting, 
philosophical discussions of the purposes of children's services, questions about the key 
indicators of service, and doubts about the feasibility of measurement were aired. 
Because people had not actually participated in such data collection, there was a need 
for theoretical background and explanation which was provided by Zweizig. It was 
necessary for the group to look at and reach consensus on terminology and service 
questions. Considerable discussion about the use by adults of children's materials and 
services occurred in examining each measure. It reflected the knowledge that children's 
librarians serve children both directly and indirectly through service to parents and 
other adults. 

At the second meeting, methodology was emphasized as proposed adaptations 
were discussed. The measures were then performed in 20 libraries, using a draft manual 
of adaptations of measures prepared by Zweizig. In addition to reporting results of the 
measurement, librarians were asked to report on the experience of obtaining each data 
element and overall reactions to the data collection process. These reports were used by 
Zweizig to identify problems and to propose solutions. At the third meeting, the data 
were presented, and discussion and recommendations for changes in the data collection 



Zweizig, Braune, k Walty 



SCHOOL OF LIBRARY AND INFORMATION STUDIES 



forms occurred. In some cases, individual libraries collected the data as the group had 
decided, but with slight additions so that they could be interpreted for the group study 
and also in a way which reflected the individual library's needs. Their ability to do 
this supports the adaptability of the output measures methodology and the usefulness 
of the forms used. There was not enough time in the third meeting to thoroughly discuss 
the data and to draw conclusions about what the information told libraries, 
individually and collectively, about the state of children's services. For this reason, it 
is recommended that a fourth meeting of participants would be important. 

This is a progress report. The data included should be viewed as results of the 
process, not as base-line data, although those Wisconsin libraries repeating the 
collection will be able to use it to build on and for comparison. But because this is a 
testing experience-to see how the forms adapted for children's services really worked, 
to see how the librarians and their staffs used the forms and the experience, and to see 
how children respond to such data gathering-some changes in the final product have 
been recommended and incorporated in the body of this report 

The children's librarians who participated in this test project have indicated an 
extremely positive response to this process. Many of them have plans to repeat the 
process. It is too early in the process and date from a single measurement period are too 
limited to come to conclusions about children's services in Wisconsin. However, on the 
basis of the enthusiasm fo; this method of collection expressed by the participants, it is 
hoped that in 1986 systematic collection of data for children's services will begin. 



Introduction to the Manual 

This document is both a report of a test of adaptations of Outpu Measures for 
Public Libraries for the measurement of children's services and a manual for children's 
librarians to use for future date collection. The appropriateness of the measurement 
procedures in the manual is the result of the field testing by and recommendations of tii 
children's librarians in Wisconsin. 
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The material in this manual is intended to supplement the OMPL manual, Output 
Measures for Public Libraries; A Manual of Standardized Procedures (Douglas Zweizig 
and Eleanor Jo Rodger; Chicago, American Library Association, 1982), The numbers of 
the measures and the page numbers references in this publication are to that manual. 
The OMPL manual should be consulted for fuller explanation of each of the data 
collection procedures.* 

Output measures are standardized indicators of the services a public library 
provides to its community. In order to aid comparison among libraries and across time, 
they are expressed as ratios to adjust, for example, for different sizes of populations 
served* The figures which form the output measures ratios are referred to as data 
elements. Juvenile circulation per juvenile capita, for instance, is made up of the data 
elements of Annual circulation of juvenile materials and Juvenile population in the 
jurisdiction served. The data collection procedures in this manual are for the collection 
of the data elements. The resulting output measure is produced by dividing one data 
element by another. 

The determination of definitions for measures of services involves many 
compromises, and many difficult definition problems remain. Probably the most 
problematic is that of the population served. Even though libraries receive the greater 
part of their support from a locality, a significant proportion of their service may be to 
residents of nearby communities or unincorporated areas. The considerations that 
resulted in the use of the population of the primary jurisdiction as the data element 
were that this figure would be used for other municipal services, that it was the figure 
most meaningful for the primary hinders, and that it was the most unambiguous of the 
definitions considered. The addition of a percentage of the service that is to 
non-residents (in Table 2) aids in the interpretation of the per capita measures. 



* See also:Nancy Van House et al Output Measures for Public Libraries; A Manual of 
StandardizedProcedures, Second edition. Chicago: ALA, 1987. 
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In order for data to be interpreted, they must have a basis tor comparison: last 
year's data, the target level set in the library's objectives, or the like. Next year, when 
this data collection is repeated, librarians will be able to observe the directions in 
which the measures are moving. At this time, the only comparison set of data 
available is from a national gathering of output measures reported in 1983.* In the 
comments on the measures from this project, these national data for total libraries (in 
contrast to data for children's services) are used to provide some context for 
interpretation. 

The librarians who participated in this project moved from an initial uncertainty 
about the feasibility and usefulness of the output measures process to an enthusiasm 
tempered by experience with the complexities of measurement. They asked that a 
quotation from an English judge on the quality of statistics from India be included in 
this report: 

The government are very keen on amassing statistics. They collect 
them, add them, raise them to the nth power, take the cube root, and 
prepare wonderful diagrams. But you must never forget that every one 
of these figures comes in the first instance from the village watchman, 
who just puts down what he damn pleases. 



1 Douglas L. Zweizig, "Any Number Can Play; The First National Report of Output 
Measures Data," Public Libraries, 24: Summer, 1985, 50-53. 
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Measures Used in this Project 



Measure 1: JUVENILE CIRCULATION PER JUVENILE CAPITA: 

Annual Circulation of Juvenile Materials divided by Juvenile Population. 

Measure 2: IN-LIBRARY USE OF JUVENILE MATERIALS PER JUVENILE CAPITA: 
Annual in-library use of Juvenile Materials divided by Juvenile Population. 

Measure 3: JUVENILE LIBRARY VISITS PER JUVENILE CAPITA: 

Annual Number of Juvenile Library Visits di\ ided by Juvenile Population. 

Measure 4: ANNUAL JUVENILE PROGRAM ATTENDANCE PER JUVENILE 
CAPITA: 

Annual Juvenile Program Attendance divided by Juvenile Population. 

Measure 6: REFERENCE FILL RATE FOR JUVENILES: 

Number of Reference Transactions Completed for Juveniles divided by Number of 
Reference Transactions for Juveniles. 

Measure 10: LIBRARY REGISTRATION OF JUVENILES AS A PERCENTAGE OF 
JUVENILE POPULATION: 

Library Registration of Juveniles divided by Juvenile Population. 

Measure 1 : : TURNOVER RATE OF JUVENILE MATERIALS: 

Annual Circulation of Juvenile Materials divided by Holdings of Juvenile 
Materials. 
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Issues in Measurement of Children's Services 



Why measure children's services separately? A public library, though a single 
institution, is a complex of services for clientele with differing needs. Lowell 
Martin enumerated seven quite distinct roles played by public libraries: The 
People's University, the student's auxiliary, the children's door to reading, the free 
bookstore, the information agency, the scholar's workshop, and the community 
center. Measures of the output of the library as a whole can give only a general 
sense of the contribution of the library and cannot provide the management 
information that will allow the library to determine its effectiveness in the roles it 
chooses to emphasize. The output measures manual included Second Level measures 
for those libraries that wished to go beyond the general, initial measures. Many of 
the Second Level measures involved looking separately at services to children. 

Whv are the services provided during the summer months being ignored in this project? 
There is no question that the pattern of children's services is different during the 
summer months. In order to have a full sense of children's services, a separate study 
of output measures during the summer would be required. This project is an 
exploration of the feasibility of adapting output measures to children s services. 
After the measures are tried this spring and revised, those measures that have been 
successful may be used in the summer in a continuation of the project. 

What are the advantages of sampling? How do we know that sampling provides better 
information than counting everything? Most books on sampling tell how to sample 
and neglect to say why. Morris James Slonim's Sampling; a quick, reliable guide to 
practical statistics (N.Y., Simon and Schuster, 1960 [originally published as 
Sampling in a nutshell]) continues to be used as a commonsense explanation of both 
the why and the how: 

A natural and legitimate question that many readers will ask is, 
"What advantages does a sample possess over a complete count?" The 
answer in brief is that sampling often permits the executive to cut costs, 
reduce manpower requirements, gather vital information more quickly, 
obtain data that could not possibly be available otherwise, obtain more 
comprehensive data, and actually increase the accuracy of his figures in 
some instances. In addition, some sampling methods make it possible to 
measure the reliability of the sample estimates from the sample 
itself.. .. Ip31 Anyone who is of the notion that 100 per cent count 
automatically provides the true figure should try counting and 
re-counting a heap of boiled peas the size of a dunce cap until he gets 
the same total twice. lp. 14] 

If you're intrigued by this question, you may want to locate a copy of Slonim's book. 
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A Note on Sample Sizes 



The numbers indicated in the manual for sample sizes have been based on the 
assumption that the samples will be drawn from a large number of items: in sampling 
items requested for document delivery, we have assumed that a large number of items 
are requested; in sampling titles sought for title availability, we have assumed that a 
large number of titles are sought each year; and so on* If your library is small, you may 
be able to reduce the number required in the sample. To decide whether this is possible, 
you first need to think about the data element being collected, and answer the following 
questions: 

• Regarding in-library materials use: 

Are less than 2,000 items used in my library each year? Yes No 

• Regarding annual number of library visits: 

Did the library have less than 2,000 visits last year? Yes No 

• Regarding annual number of reference transactions: 

Did we have less than 2,000 reference transactions last year? Yes No 

• Regarding annual number of titles sought: 

Did users seek less than 2,000 specific items (known titles) last year? Yes No 

• Regarding annual number of subjects and authors sought: 

Did users seek less than 2,000 specific subjects or authors last year? Yes No 

• Regarding annual number of browsers: 

Did less than 2,000 users browse last year? Yes No 

ji 

• Regarding annual requests for materials not immediately available: 
Were less than 2,000 items not immediately available 

requested last year? Yes No 

For each of these questions, it's o.k. to make a guess. For example, if you're open 300 
days in a year, you might ask, "Is it likely that on an average day less than seven items 
were used in the library?" 

For any of the above questions to which you answered "No," the sample size should 
be the one recommended in the manual, generally a minimum sample of 100 items. For 
any of the above questions to which you answered "Yes," you can reduce the sample size 
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for that data element depending on how much below 2,000 you think your number is, as 
follows: 

Number of Items Number of Items 

Estimated Needed in Sample 

2,000 * 

1,000 J 

500 °° 

250 60 

For example, the library that estimates that about 1,000 items were used in the 
library last year would only need a sample of 90 items in the in-library materials use 
study. 

Suggested Order for Gathering Output Measures Data 

Most libraries have said that they plan to collect the data in a one-week period. 
If the data could be gathered in several phases, some confusion and overload of the 
staff might be avoided. A pattern has been suggested below. The pattern can be 
modified in order to best meet your needs and schedule. 

Phase 1 (Preparation)'. 

Inform staff and others involved (Board? volunteers?) of the output measures 
process youll be using and the schedule. 

Arrange for duplication of needed forms. 

Record data elements already known or easily obtained: 
juvenile population 
holdings of juvenile materials 
library registration of juveniles 
annual juvenile program attendance 

Phase 2 (First data collection): 
Conduct circulation study 
Conduct in-library materials use study 

Phase 3 (Second data collection): 

Conduct reference study (for Reference Fill Rates for Juveniles) 
Conduct study of library visits 
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Example procedures and data report 

Measure 2: In-library use of Juvenile Materials per Juvenile Capita: 
FORMULA: In-library use of Juvenile Materials divided by Juvenile Population. 
DEFINITIONS: 

OUTPUT MEASURE: In-library use of Juvenile Materials per Juvenilt Capita reports 
the estimated annual number of juvenile materials used by patrons in the library, 
divided by the Juvenile Population. 

DATA ELEMENT: In-library use of Juvenile Materials is the number of all materials 
classified or marked as "juvenile" that are removed from their shelf locations by either 
staff or users for use within the library. 

COLLECTING THE DATA ELEMENTS: 

In-library use of Juvenile Materials: During a sample week, the procedure involves 
counting the number of juvenile materials that are to be replaced on shelves or in files, 
excluding those returned from outside circulation (See In-Library Materials Use Log on 
OMPL p. 18.). If juvenile materials are shelved in a separate location, the procedures in 
the OMPL manual may be used. If juvenile materials are inter-shelved and are 
identifiable, the modified IN-LIBRARY MATERIALS USE LOG provided in this 
publication should be used. The modified IN-UBRARY MATERIALS USE LOG allows 
for an in-library materials use study for the entire library, with a separate count of 
juvenile and non-juvenile materials. 

Some libraries observed that patrons were inhibited by the survey and removed fewer 
materials than they would have without the survey or that patrons reshelved 
materials in spite of the survey. Some suggestions were that the library could put 
laundry baskets at strategic places in the library for the deposit of examined items, the 
SURVEY ALERT POSTER in this publication could be used or adapted, or volunteer 
"greeters" could explain the survey to patrons as they enter the library or children's 
room 



Zweirig, Bmine, k Walty 



SCHOOL OF LIBRARY AND INFORMATION STUDIES 



8 



8 * 




IN-LIBRRAV MI 

(Number of Hems) 



(jmcs) 



LIBHARY- 



OflTt L L 

SHEET # 



HOUR 






m 




m 




m 


m 


m 


TOTALS 




JUVENILE 




















BOOKS 


NON-JUVENILE 






















JUVENILE 




















MAGAZINES 


NON-JUYFNILE 




















PAMPHLET FILES 


v <VENILE 




















NON-JUVENILF 






















JUVENILE 




















SOUND 

Dtmoniijr.c 
HECUnUINu* 


NUN-JUYtnlLt 






















JUVENILE 




















NEWSPAPERS 


NON-JUVENILE 




















CONTDOLLEO 
MATERIALS 


JUVENILE 




















NON- JUVENILE 






















JUVENILE 




















OTHER 


NON- JUVENILE 






















JUVENILE 




















TOTALS 


NON-JUVENILE 




















TOTAL 





















a j s 



211 



SURVEY ALERT 

PLEASE! 

USE THE BOOKS, BUT... 




DO NOT PUT THEM BACK TODAY! 

Thanks. 
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Table 1 

In-library use of Juvenile Materials per Juvenile Capita 



LIBRARY % NON-RES. SCORE 
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Measure 11 : Turnover Rate of Juvenile Materials: 

FORMULA: Annual Circulation of Juvenile Materials divided by Holdings of Juvenile 
Materials. 

DEFINITIONS: 

OUTPUT MEASURE: Turnover Rate of Juvenile Materials measures the activity level 
of the library's juvenile collection. It indicates the number of times each item of juvenile 
library material would have circulated during the year if circulation had been spread 
evenly throughout the juvenile collection. 

DATA ELEMENT: Holdings of Ju venile Materials is the number of cataloged items (not 
number of titles), plus paperback holdings even if uncataloged, that are classified or 
marked as "juvenile." 

COLLECTING THE DATA ELEMENTS: 

The definition of this set of library materials will differ from library to library. 
Methods for defining juvenile materials may include using those materials shelved 
separately for service to juveniles, using those materials labeled as juvenile materials, 
or making an estimation based on the proportion of materials ordered to serve the 
juvenile population. 
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Table 7. 

Turnover Rate of Juvenile Materials 
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Observations on the Data Collection Experience 

• We had fun during the in-library materials use week. It brought a great deal of 
attention to an area neglected by staff and public alike. 

• All in all, our staff was supportive and conscientious. 

• "Collecting the data during the sample week has led us to be more observant of 
patrons and how they use library materials." 

• "We enjoyed doing the output measures and feel our board of trustees will be 
interested in the results, particularly in-library use of materials and library 
visits." 

• "I enjoyed the experience! By doing this study, I was made more aware of the 
statistics that need to be kept and am already thinking of ways to use these 
figures. I feel that you really have to go through the procedure once before you 
are ready to ask intelligent questions." 

• "We gained a more practical political outlook on statistics. We gained insight 
into how we could cut our daily record keeping while gaining more accurate and 
useful statistics on a twice-yearly basis." 

• "From the data collected, we could also calculate measure #5 from the manual: 
Juvenile Reference Transactions/Juvenile capita = 032." 

• "Think that doing it again would go easier as patrons would begin to think 'Oh, 
that again' and know better what to do." 

• "It was a lot less painful than I thought it would be. I found the data collection well 
worth the effort. Based on the output measures figures I have obtained, I'm motivated 
to set up some measurable objectives or goals for the next year and am eager to perform 
output measures a year from now to see how things liave changed." 
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Managing Children's Library Collections 
Through Objective Data 

Craighton Hippenhammer 

Top of the News, Spring 1986, p. 309-313. 



Most librarians who serve children enjoy selecting materials. They expend 
considerable effort at perfecting their selection skills because they want their library 
collections to have the richest choicest and most useful resources for their young 
patrons. But excellent collections do not develop through good selection alone and will 
never happen without effective collection management. 

The main collection management tool currently available to children's librarians is 
the recommended list Lists can be a helpful double check of one's own expertise and 
currency; those published by knowledgeable practitioners and academicians are 
invaluable for checking the quality of one's collection and to insure that important 
titles are represented. But lists cannot tell librarians whether the listed materials 
should be in their particular collections. Children's collections must be analyzed as a 
whole to assess whether they are meeting the needs of the patrons they serve and 
following the goals and policies of the institutions to which they belong. 

Ways of conducting comprehensive collection evaluations are now being developed 
within the profession. Most involve the gathering of statistical data; it is the 
monumental clerical task that this presumes that boggles the minds of collection 
managers. While computer help looms on the horizon for some, there are still 
techniques such as sampling and periodic measurement that librarians with 
nonautomated collections can learn to simplify the task. 
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Library directors are finding it valuable in financially difficult times to be able to 
show objective, verifiable facts to prove to their hinders fiscal accountability and 
documented results. Many of them are exdted about ways of measuring statistical data 
called output measures that may help them in documenting results. Children's 
librarians may wonder what's in it for them except for the threat of additional 
pressure to produce more than ever— more programs per staff hour, more reference 
questions answered, etc. The answer is this: output measures can give children's 
librarians additional information to help them to get better control of their collections 
and to make better selection and weeding decisions. Finding new ways to examine one's 
children's library collection via statistical comparisons invariably proven to be 
fascinating to the professional because of the new insights it makes possible. 

Circulation Statistics 

The most widely known and widely kept output data are circulation statistics. 
Circulation figures, in a rough sort of way, reflect library policies and the activities of 
librarians. Wise collection building, consistent, high quality public relations, and 
effective readers' advisory usually have a positive effect on circulation. But changes in 
demographics, community-organization activities, patron interests and habit patterns, 
or even the weather can also affect it. So while there are some circulation-influencing 
factors that librarians can affect, there are others that prove more difficult or 
impossible to control. 

Even so, circulation statistics can prove useful to the collection manager, especially 
when compared to other data or to changes over time. To be the most helpful in 
collection management, circulation statistics should be kept by collection section, i.e., 
separate data for picture books, fiction, nonaction classifications (perhaps by Dewey 
hundreds but better yet by Dewey tens), type of media, etc. This makes it possible to 
examine the collection in finer detail and to draw conclusions about specific collection 
areas. For example, here are some situations that objective data comparisons might 
uncover 
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Comparing Circulation Year to Year 

Circulation has increased over the years in all sections except for the 500s. Why? 
Fiction circulation decreased under the last children's librarian but has been increasing 
under the current one. Is the current one doing something right? 

Comparing Circulation to Patrons Served 

If circulation is decreasing sUghtly but the child population is decreasing 
significantly, the circulation rate per patron is increasing. This is a useful statistic if 
there ever was one. Perhaps the preschool population is increasing but picture-book 
circulation is decreasing. Why? If the circulation of children's books remains the same 
while circulation of adult books increases, are children checking out more adult books? 
If so, why? 

Comparing Circulation to Number of Materials Owned 

Also known as turnover rate, this figure will be different for each collection section. 
Comparing turnover rates within a section yields helpful information. For example, if 
one's picture-book section's average turnover rate is nine circulations per picture-book 
volume per year and title X circulates an average of only once a year, then unless aimed 
at the special reader, title X is either a candidate for weeding or for resurrection via 
book talks or story hours. Comparing turnover rates between sections will at times also 
prove valuable. Picture-book section turnover rates can be expected to be two to four 
times that of the juvenile fiction section. If the picture-book section turnover rate is ten 
times that of the fiction section, it would seem wise for the juvenile collection manager 
to either increase the size of the picture-book section or to decrease the size of the 
fiction section. 

Measuring in-house use of materials may prove useful for some libraries. This is 
especially true of collections where reference materials get heavy use or in children's 
rooms that hold materials for projects that several classes or other groups must use ir. 
the library. 



UNIVERSITY OF WISCONSIN— MADISON 



Hlppenhammer 



Availability Statistics 

Another method for evaluating collections is measuring the availability of 
materials. One way to measure the availability of materials is through survey 
interviews. Patrons arc asked upon exiting the library whether or not they found the 
subjects or titles for which they came. The percentages of yes answers are known as fill 
rates. 1 

The patron-interviewing technique, unfortunately, measures too much. It confuses 
and confounds measurements of collection performance and patron response to the 
collection. Not only does the fill rate measure whether the material is actually in the 
collection, but it also measures patrons' abilities to find materials on their own, their 
abilities to find materials with the help of a librarian, ease of access (library 
architectural impediments and collection arrangement peculiarities) and the level of 
staff helpfulness. All of these factors are measured together and cannot be separated. 

Many subjective factors weaken the measurement of availability when the 
interview technique is used. Adult patrons often do not know what they want, are not 
able to verbalize what they want, or prefer to avoid asking for help. Children have 
all of these difficulties and more: their inexperience, their strong desire to please 
adults and their penchant for settling for less than what they need. Such factors will 
greatly skew interview results. Expect juvenile fill rates via interview to be vei h 
and more than likely inflated. Remember, juvenile materials are less available, ut 
general, than adult materials. 2 It is difficult to see how fill rates gathered in this torm 
can be accurate enough to be useful in evaluating collections — especially juvenile 
collections. 

Despite the muddled nature of this kind of materials "availability," the interview 
technique has enjoyed considerable support. However, this support is being questioned 3 
and some librarians are now recognizing the need for refinement or even a new approach. 
It is interesting to note that the 1984-85 Wisconsin LSCA project on children's services 
output measures 4 did not measure title and subject fill rates due to practical reasons, i.e., 
the difficulty of arranging for and gathering the survey data. 
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To measure availability in a way that makes it more useful and less muddled, try to 
measure fewer factors at a time. One alternative method is t*> divide "availability" 
into "external availability" and "internal availability." Extet^al availability 
measures those materials that are unavailable in the collection at the time of the 
request. Track the unavailability of materials by keeping an Unfilled Requests Log. 
Whenever a patron request cannot be answered, note the title or subject in the log and 
the reason the request could not be filled: the item is "checked out" or the library "does 
not own." Accumulate all entries at the end of the month and tally up those that recur. 
Entries in this monthly Accumulated UnavaUability Data Log that appear over a 
period of time or with sudden intensity are candidates for purchase. Some librarians try 
to spot such trends informally as they run across them but it pays to accumulate the 
data in written form, especially if there is more than one staff member working in the 
children's area. 

Internal availability involves gathering data directly from the collection and will 
answer the question: Of all the titles in a particular collection section, what percent are 
on the shelf, available for circulation at a particu' ^r point in time? The technique as 
explained by De Prospo 5 involves sampling the shelflist and physically checking the 
shelves for the titles picked. This approach has two strong points in its favor. First, it 
gets rid of the subjective uncertainties of surveying people and secondly, it provides an 
excellent method for evaluating duplication rate within collection sections. 

Collection managers know the importance of duplicating popular titles. The more 
unwanted titles patrons have to search through to get to the ones they want, the 
greater the chance they will end up choosing fewer titles. If frequently used titles are 
duplicated heavily and little-used titles are weeded, circulation will increase even 
though there is a decrease in the total number of titles. Assuming adequate collection 
size, as the rate of duplication increases, both availability and circulation will 
increase until duplication saturation hits. Most juvenile collections are a long way from 
too much duplication. 
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Measuring internal availability rates will uncover where duplication is needed. If 
two copies of a book are owned, the title will be available more often and to more 
patrons than if just one copy is owned. This fact will be reflected in the 
internal-availability rate (number of titles found on shelf divided by number of titles 
owned) of the collection section. Collection managers can use this data to experiment. If 
the availability rate of one collection section is low, the children's librarian might try 
ordering additional copies of fast-moving titles in that section and weeding 
slow-moving ones. If the number one fault of public library collection managers is buying 
too many titles and not enough copies like some believe, then an excellent use of internal 
availability rates would be to raise the duplication rate overall, thereby raising both 
availability and circulation for the whole collection. 

Most of the collection management techniques described so far can be done manually 
and in some libraries where objective measures have been seen as important, they have 
been for many years. Computers, of course, can make some of the work easier and can 
provide more detailed data that would be difficult or time consuming to gather. Now 
that computer help is here, however, that often-heard but unfortunate assertion "We're 
too busy to gather statistics" will be seen for its real meaning: Objective measures are 
not important. 

There is a fear among children's librarians that the use of objective data will 
increase the qv entity of popular materials at the expense cf quality, usually 
less-popular, materials. Collection management statistics are simply tools; they 
provide additional information that collection managers can use to see their collections 
from additional viewpoints. Such an approach does not have to conflict with a concern 
for quality materials. After all, a good review gets a title into the collection, but a good 
circulation track record of a title only gets the collection additional copies or 
additional titles that have been recommended. If collection quality suffers at a 
particular library, it is the fault of library policy or its ineffective implementation, not 
the fact that statistical tools are used. 

Robbins-Carter and Zweizig have said that "the central task of collection 
evaluation is to assess the relationship of the offered collection to the moving target of 
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user needs." 6 With statistical tools at the collection manager's disposal, collection 
weaknesses are easier to discern and library collections will become more finely tuned to 
their users. 

Interest in output measures has grown recently in many quarters. Concerning school 
library collection building Jacqueline Mancall and Christopher Swisher state: 'There 
is no longer any place for the uninformed collection manager. Unique usage factors and 
curriculum plans should determine all collection activity. Media specialists should 
incorporate local evaluation studies of materials' use into collection decision making."' 
The Public Library Association is currently developing standards based on output data 
and is encouraging data collection and output measure surveys. The American Library 
Association is publishing books and manuals on the subject. 

Subjective collection management techniques like list checking and evaluation by 
personal impression are no longer enough. Librarians who serve children should add 
objective output data techniques to their collection management skills if they have not 
already done so. Using statistical data to increase the factual base of collection 
management will give librarians the tools to know and interpret their collections in 
new ways and provide evidence for convincing administrators of outstanding 
performance. Improved library services will be the result. 
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IV. Structured Discussion as an 
Evaluation Technique 



Introduction 



Previous chapters have discussed a number of evaluation techniques, most of 
which involve counting things. (Even interviewing can produce countable results, 
though the processes described by Eliza Dresang and Lynn MacDonald do not 
necessarily lead to counting.) Many librarians feel that what they do is not 
measurable in numerical terms. Since most of us come from humanities backgrounds, we 
value the personal, aesthetic, intellectual, educational, even inspirational qualities 
that library users may find in library materials and services. Public library work is 
characterized by service to individuals who come to the library voluntarily, and 
public librarians are loathe to reduce individuals to ciphers, particularly children. 

Nevertheless, the quality and effectiveness of services and programs need to be 
evaluated for the purposes of planning and decision-making. This chapter, then, is 
about an evaluation method that takes full advantage of the marvelous diversity of 
humankind and encourages individuals to express their opinions, reactions, ideas, and 
desires: Structured discussion. 

What is meant by this is not the usual staff meeting where news is exchanged and 
problems discussed. Undoubtedly evaluation sometimes takes place in such meetings, 
but here we are dealing with meetings called specifically for the purpose of 
evaluation. Discussion for evaluation requires a structure appropriate to the situation 
and the kinds of decisions to be made, a discussion leader or facilitator who can keep 
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the discussion on track without imposing her own views, and group members who 
are prepared to discuss the issues, who have done their homework, in other words, and 
have a common ground upon which to base their discussion. 

In the articles that follow, the authors focus on discussion for the evaluation of 
children's books in a variety of circumstances. The ideas and methods proposed are 
well suited, however, to many other situations. Discussion may be used to evaluate 
policies and procedures, management practices, goals and objectives, activity 
programs, services, staff training, and facilities. In fact, structured discussion can even 
be used to decide what ought to be evaluated! Discussion can be a valuable adjunct to 
statistical kinds of evaluation. Fur instance, summer reading programs are typically 
evaluated by such measures as the number of children who signed up for the program, 
the number of children who completed the program, and the number who attended 
special activities. To add perspective to these counts, it can be helpful to discuss the 
observations of the program with a few of the participants, including clerical staff, 
teachers, parents, and children. A more complete picture of the program will emerge, 
increasing the likelihood that when changes are made, they will be meaningful 
changes. 

We sometimes think of evaluation as a process that is designed at the beginning of 
a project and carried out at the end of the project. Some writers call this "summative 
evaluation" because it "sums up" the thing being evaluated. In actuality, it is often 
helpful or even necessary to evaluate a project before its midpoint or to try a new 
procedure for a period of time and evaluate it before adopting it in a final form. This 
kind of evaluation is called "formative evaluation" because it treats activities as 
on-going processes susceptible to change in midstream, forming and re-forming 
themselves. Many aspects of children's librarianship are constantly shifting shape 
rather than coming to a discrete terminus; formative evaluation is particularly 
appropriate for such activities. Discussion can be a convenient method of formative 
evaluation because it does not require statistical apparatus, questionnaires or large 
samples. During a year-long LSCA project, for example, it would be better to have a 
structured discussion at the end of the fifth month than to wait until the year was up 
to evaluate the completed project. If the project as planned is not meeting its 
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objectives, far better to know about it in time to make change* than to risk the quality 
of the entire project. 

The articles that follow take up many of the fine points to be considered when 
organizing and carrying out discussions. The first article discusses the need for 
discussion in light of the nature of reading. In the second article, Kathleen T. Horning 
writes about discussion as part of the process of selecting new books, in addition to 
using published reviews. Books may also be discussed at the time of a complaint or 
challenge to their presence in libraries. Dianne Hopkir* shows how discussions for 
reconsideration differ from other kinds of book discussion. Eliza T. Dresang then 
explores the unique qualities of discussion for awarding distinctions and medals, 
particularly the processes used by the Association for library Service to Children. 
The chapter concludes with the guidelines used at the Cooperative Children's Book 
Center at the University of Wisconsin— Madison and a commentary on how those 
guidelines function at the CCBC and how they may be used in contexts other than book 
discussion. 
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Book Discussion as an Evaluation Tool 



Holly Willett 



Children's librarians are familiar with book discussion for a variety of evaluative 
purposes. Many of us are involved in local or regional book selection discussions; we 
participate on book award juries; some of us have been involved in discussions 
reconsidering a book that has been challenged. Probably all of us have discussed books 
for the pure joy of it. Professionally and « -rsonally, we spend a lot of time talking about 
books, deciding their quality or estimating their usefulness to library users. We 
probably haven't spent much time thinking systematically about book discussion itself, 
though, and in recent years, many of the large library systems have given up the 
regular book discussion meetings that characterized children's services ten or fifteen 
years ago. 

I wish to remind readers of the things they already know about book discussion as a 
tool for evaluation but put the knowledge into a more cohesive framework. I will deal 
with the topic in a general way. Kathleen Horning, Dianne Hopkins, and Eliza 
Dresang will dipcuss specific contexts of discussion: evaluating new books, reconsidering 
challenged books, and conferring awards and distinctions upon books. While the four of 
us are concerned about evaluating books through discussion, many of our comments would 
certainly apply to discussion of film, video, computer software, magazines-virtually 
any medium of communication-as well as many other aspects of children's services. I'm 
going to begin with a specific example of an experience which will be familiar to the 
reader. 

One of the courses I teach is a survey of children's literature. Towards the beginning 
of each semester, I ask my students to study two picture books, The Three Robbers by 
Tomi Ungerer ( Atheneum, 1962) and Ben's Trumpetby Rachel Isadora (Greenwillow, 
1979). Inevitably someone in the class will say that she doesn't like The Three 
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Robbers because it teaches children that it's okay to steal. The first time this happened 
I was very surprised— I shouldn't have been, but I was— since I've always relished the 
book and cheerfully used it with preschoolers. Generally, most of the class disagrees 
with the objecting student. When we proceed to talk about Ben's Trumpet, the student3 
initially have a lot to say about the graphic emotional power of the illustrations and 
their interaction witl( the text. Finally, I will ask if anyone can see why some people 
have objected to the book as being racist. Nearly all of my students are white and there 
has never been an African American in the class, so sometimes students take a little 
while to think about this. Eventually, someone will point out that the image of Ben's 
family is rather negative. Other students will disagree that blackness is important in 
the context of the story. Usually there is a lively discussion of whether either of these 
books should be presented to children, and if so, how. 

Surely the reader has encountered someone who disagreed radically with your 
point of view about a book or remembers at least one instance in your undergraduate 
career in which an irritated professor dismissed a hapless student's maladroit 
interpretation of a major work of literature. Perhaps you have read a glowing book 
review in one journal only to find that another reviewer in another journal has panned 
the same book. More often, you find that the other opinion is not radically different but 
simply presents an aspect of the book that you had not observed. It is these 
disagreements and variant perceptions which make it necessary to use book discussion 
for the purposes of evaluation, particularly in situations where some kind of consensus 
or group judgment is needed. 

Most of us probably accept the notion that the source of the disagreements or 
differing points of view springs from each human being's uniqueness. The process by 
which we arrive at our differing perceptions is actually rather complex and 
fascinating. It is intimately bound to the process of reading. As I've been reading about 
criticism and the effects of media, I've observed that there seems to be a convergence of 
thought among cognitive psychologists and literary critics. They seem to be telling us 
that reading is a process of negotiation or construction, in various meanings of those 
words. In what follows, I'm going to talk about reading in general, not just from a child's 
point of view, and my examples will not all be from children's literature. 
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A reader has on the one hand a message from another person or persons, (the author 
and/or illustrator) who is most likely not an intimate acquaintance of the reader, and 
on the other hand, the reader has a set of tools for deciphering that message. The 
reader's toolkit includes a knowledge of written and pictorial symbols, a knowledge of 
how texts are organized, a knowledge of Ufe based on the reader's personal experience, 
and perhaps some knowledge related to the book but not part of the book, such as 
knowing the biography of the author. Most importantly, the reader has the skill to use 
the other tools to make sense of the message. As a reader works through a text, she 
decodes the written alphabetic symbols plus the graphic symbols of any illustrations 
and constructs meaning out of the text by using her tools in a strategic manner. She asks 
hsrself a series of questions, such as, what do I know about this subject? how do I read 
this kind of text? does this make sense? why did the author tell me this about the 
character? whaf s going to happen next? do I need to go back and reread the previous 
page? does this story fit what I know about how real people act in such situations? All 
the time, the reader is constructing a personal mental image of the text that grows and 
changes as she reads. 

You can easily see how the contents of readers' toolkits could differ greatly from one 
reader to another: Family background; life experience; when and where and how you 
learned to read as well as your inherited intellectual abilities all have their parts to 
play in your reading. The nature of one's toolkit can determine what one reads and how 
well one reads or whether one reads at all. Children's toolkits are smaller than those 
of adults, since adults have not only had time to acquire more tools and fancier tools, but 
also have the cognitive development to use their tools. Adults have made frameworks 
from past reading experiences, such as patterns that stories frequently use. We have 
larger vocabularies, greater experience of the world, greater experience in making sense 
of a variety of texts, and sometimes a lot of education in how to construct meaning out of 
texts, perhaps even a course in children's literature. One of the major difficulties in 
evaluating children's books is that adults must attempt to judge the fit between what 
we estimate to be in children's reading toolkits and the books that authors, illustrators, 
and publishers provide for children. And very often we disagree among ourselves about 
how well a book will suit children because attempts to gauge the fit between book and 
child are pure speculation, based on our differing experiences with children in the past, 
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which may have nothing to do with the children who will actually read the book. We 
base our judgments of the fit between book and child on a non-random sample of 
children. 

Authors and illustrators, for their parts, make assumptions about what their 
readers know, assumptions about the contents of their readers' toolboxes, if you will. 
Now we have all heard authors who claim that they write for themselves and don't 
consider their audience at all. Frankly, I'm skeptical of this claim. If you're only 
writing for yourself, why not just keep a journal? Why go through the agony of 
publishing? I suspect that most authors for children are to some degree both conscious of 
and sympathetic to their audiences. I will let Katherine Paterson speak for this point 
of view: 

My aim as a writer is to engage young readers in the life of a story that 
came out of me, but which is not mine, but ours. I don't just want my 
readers' time or attention, I want their lives. I want their senses, 
imagination, intellect, emotions, and all the experiences they have 
known breathing life into the words upon the page. I hope to do my part 
so well that young readers will delight to join me as co-authors. 
(Kirkpatrick, p. 603.) 

Paterson would agree with the cognitive psychologists that reading is not a passive 
reception of whatever comes. It's hard work. 

Writers leave cues and clues for readers, like Hansel leaving white stones and 
bread crumbs behind him. And like the bread crumbs and white stones, some cues are 
more effective than others. Individual books have their individual ways of requiring 
the reader to co-author the text, and genres also have characteristic ways of provoking 
reader involvement with the text. Some genres even have ways of deliberately 
misleading or confusing the reader. For example, mysteries are guessing games in which 
the reader must negotiate a path strewn with false clues in order to construct both the 
story of the crime and the story of solving the crime. In many fantasies the author sets 
the reader down in a secondary world where the reader must infer its parameters and 
possibilities from events, setting, dialogue, and internal monologue. 
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In order to participate fluently in this joint venture of text construction, readers must 
have well-developed toolkits and must have learned how to use them to the point that 
they are just barely conscious of doing so. If one needs to be consciously decoding a 
sentence word by word or forcing oneself to monitor one's reading moment by moment, 
reading slows, down, becomes tedious, and loses its savor. Many children and adults are 
in this predicament. One of my acquaintances is dyslexic and hardly ever reads fiction 
because she simply cannot spare the time it would take away from the reading she must 
do for work. She has compensated for her perceptual difficulties by being a slow but 
very comprehending reader. Those of us who read more quickly may be barely conscious 
of what we're doing as we read. And this has its drawbacks, for it means that we are 
not fully aware of our responses or the stimuli that trigger our responses to what we 
read. We do not always recognize the cue in the text laid down by the author. We build 
meaning so quickly that most of us can't recall the entire process, not even through 
conscious effort. 

I draw three collusions from this long-winded exegesis on reading. First, all of us 
are handicapped readers. Second, when we read books we often find ourselves in them 
at least as much as we discover something new. Third, evaluation of any particular 
book should take into account the means by which the author invites readers to make 
sense of the text. Each of these points constitutes something of a dilemma for those of us 
engaged in ev^ating books, but I believe that book discussion can help us. 

When I say that we are all handicapped readers I mean that we all have 
limitations on our reading ability. All of that experience which opens our lives to the 
vicarious potentialities of books also limits what each of us is able to take from 
reading. We may not recojjnize a particular word; we may not have read the Bible or 
Grimms' fairy tales and will miss allusions to them, or we may not have read James 
Joyce before and therefore don't really know what to do with him. Furthermore, we 
may have such limited experience of the world that a book set in a foreign country may 
be incomprehensible because the characters' motivations are not clear to us. In many, 
many ways we may encounter what have been termed no-fault communication accidents 
in our reading. 
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In April 1989 1 attended a conference of literary scholars which provided me with 
an example of the limitations of experience among highly sophisticated readers. A 
major concern among the conferees was the increasing plurriity of English language 
literature. That is, we now have major writers in the English speaking countries v 10 
are members of aboriginal cultures, such as Maoris in New Zealand and Native 
Americans in the United States. They are writing novels about the intersec tion between 
their culture and European culture in styles that reflect the two ways *tf thinking, a 
situation which requires that readers know something about both cultures. Salman 
Rushdie's work is a case to point. The scholars felt at a loss to deri with the situation 
because k seems impossible for anyone to know enough about all cultures to be able to 
comprehend the vwlety of cultures represented in English language literature today. 

If schollfs are having proWems, what will happen to children? Children'* 
librarians are concerned; and rightly so, that there be more children's b&aks by and 
efeotrt minorities. Some of us believe that if we had moie children's books about 
minorities and about cultural dashes, today's children would have fewer r eoblems 
dealing with a pluralistic literature when they become adult readers, but we have no 
way of predicting what children will learn about other cultures from books. 

The point is, no <nu has the perfect reader's toolbo> . if we could even discover witat 
it might contain. Whether you are a child or a scholar or something in between your 
background is necessarily limited, and you do not recognize everything an author wants 
you to understand in her book. Nor wili you ever be abb* fr# feocMfer all af your responses 
to a text after you have finished reading it. 

Therefore, if those of us involved in book evaluation are to perform adequately, we 
must pool our resources. Th re are various ways that we can do this. We can gather up 
all th* book review* and all the critical essays, and we will have part of the 
discussions. But this is essentially a one-sided affair. You can't have an exchange with 
a book review. And exchange among those who have read the same book is vitally 
important to the evaluative process. What one person says may trigger a memory in 
another person or integrate two confusing passages in the mind of a third. Discussion 
may bring out allusions to other books or provide information that wiJl clarify 
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motivation of characters. One's nagging but unnameable feelings are finally given 
shape by someone else's comments during discussion. Reviews can sometimes provide 
such amplifying insights, but it is in the back and forth of book discussion-where you 
say something, and I respond, and you say something else, and a third person joins in 
with a question-that a variety of views and understandings can be aired with the 
possibility of responding not only to the verbal content but also the bod- language of the 
discussants. 

Through the process of sharing our experiences of reading a particular text, we 
create much fuller mental images of the text as a whole. A sense of the wholeness of a 
book is a necessity for rendering judgment of it. But wait, if making sense of a text is such 
an individual affair, how can there be any reasonable group evaluation of it? The 
second implication of the nature of reading was that in reading we so often find 
ourselves. In fact, many readers use their reading to do this quite deliberately. Because 
we create the text as we read, we not only find ourselves in it we also add things of our 
own making to it, things not intended by the author. If we take Katherine Paterson's 
word for it, it is desirable that readers make these "additions" to texts, but then there 
can be no incorrect interpretations, no incorrect evaluations, everyone is right and no one 
is wrong. What we discuss may not be so much a text but ourselves, selves that books 
sometimes allow us to encounter in perhaps more organized and coherent ways than we 
usually do. If we put our individual constructions with their various assumptions, 
interpretations, and creations into one big development, so to speak, won't we just make 
a jumbled, incoherent mess, perhaps a literary strip mall? Sometimes this happens, 
sometimes we leave a discussion feeling as if nothing is settled, and we have more 
questions than answers. 

This is not necessarily a problem. It is good to have questions because they keep us 
thinking. It is good to remember that one evaluation discussion is not the final word on 
any book. Consider the Newbery Medal books, supposedly "distinguished 
contributions" to American children's literature. Each Newbery book was judged to be of 
lasting value at the time it was given the award, but many of the early winners are no 
longer read or discussed, not even by children's literature scholars. We should not fault 
our foremothers for being poor guessers. Our own guesses about books may not stand the 
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test of time, either. The process of discovering which books mean the most to readers 
and offer the most opportunities for many people to find valuable aspects of themselves 
in literature is a process of constant revaluation. The value of the Newbery and other 
distinctions may not be in the books themselves but in the level of discussion that they 
occasion, inside and outside the award committees. 

Even though we would shy away from saying that individual interpretations of 
literature might be wrong, it is safe to say that some evaluations may be more valuable 
than others, in the sense of being more helpful in particular situations. We've all known 
individuals whom we consider to have superior literary insight. Often these people 
have an ability to integrate various aspects of a book and create a whole that they can 
articulate to others — they can create a vision and share that vision— illuminating new 
possibilities for the rest of us. When these gifted people share their sensitivities with 
a group in a book discussion, the listeners' ability to go deeper into a text (and into 
themselves) is enhanced. 

It is the learning that takes place during book discussion that leads me to add 
something parenthetically about book discussion and performance evaluation of 
children's staff. Book discussion is one way of discovering the strengths and weaknesses 
that staff may have in their knowledge of children's literature in particular and 
literature in general. Through carefully chosen books and a well-designed framework 
for discussion, one may not only diagnose problems but also begin to cure them. 

To go back to my question about the possibility of evaluating books through 
discussion when readers create so much of the book: Part of the answer may lie in 
having a framework for the discussion, knowing what you have to decide and designing 
a mechanjsm for reaching a decision. Such frameworks or patterns would vary according 
to the context of the evaluation, so I will not say much about them here as my 
colleagues will talk about various contexts of book evaluation through discussion, Book 
discussion also requires tactful leadership which allows disagreement and in fact sees 
conflict as a necessary ingredient of creative problem-solving. A book discussion leader 
should be capable of integrating disparate views without imposing her own on the 
group as they seek resolution. Another part of the answer to my question is that the text 
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itse lf i S always available as a referent for discussion, a common ground around which 
the conversation revolves. While readers may disagree about meaning and value, they 
still have the words of the text in common, and after all, part of the fascination of book 
discussion is the fact that people do differ in what they have to say. 

The third implication of the reader-as-coauthor theory is that the mechanisms 
and techniques the author uses to invoke the reader's constructive abilities should be 
part of the evaluation. Children's librarians are used to talking about plot, style, 
setting, point of view, characterization, tone, and unity-all those elements of 
pedagogical or structural criticism we were taught in high school and college. In 
addition to these aesthetic qualities, we frequently talk about whether children have 
enough experience or vocabulary to appreciate one book or another. This implies that 
children', librarians are very aware of the question of audience; so, what I'm suggesting 
is not all that new. I simply recommend that we become more conscious and systematic 
at including the reader-in-the-book as part of book evaluation. 

Vocabulary and experience are not the only assumptions authors make in devising 
techniques to hook their readers. For example, books for young adolescents are 
frequently written in the first person, betraying an assumption that these young people 
are self-absorbed and wM most easily identify with the protagonist through the "I." 
Consider how picturebooks for young children use illustration with text to deliver a 
complete story, assuming that the reader/viewer has the intellectual skills to 
integrate the two modes of storytelling. 

Authors also have authorial voices, which give the reader a sense that a 
particular individual is speaking directly to her. The author's real personality 
actually may have nothing to do with the author that the text implies, an author who 
wants the child reader to understand that the author is on the child's side. To pick a 
familiar example, consider E. B. White. Throughout his writing, whether letters, 
essays, or books for children, White maintains a very characteristic tone of genial, 
ironic narration, poking affectionate fun at his characters. In Charlotte's W«b, many of 
the adults are comic stereotypes to whom the child reader can feel pleasantly superior. 




White seems to say, "See, I'm your ally, I know how it feels to be a child/' This same 
sense of intimacy is part cf his works for adults. 

Authors also leave gaps in their stories that the reader must fill in because the text 
doesn't explicitly tell her. For example, unless the reader figures out that Max 
imaginatively creates the place Where the Wild Things Are, the reader will not fully 
appreciate Max's achievement in dealing with his anger toward his mother* 

Evaluators might ask themselves, what is this author doing to make it easier for 
children to become involved in this book? Are these tactics successful? Are they 
appropriate to the intended audience? These speculations can be explored most fully in 
a group book discussion context where the variety of perceptions and responses can help 
readers make more sense out of their reading. Even though reading can be viewed as an 
attempt to make sense out of a text, every such attempt is not successful. Book discussion 
can improve the success rate and give evaluators for more information for the 
decision-making that must frequently take place. In closing, I would like to return to 
the discussion of Ben's Trumpet and The Three Robbers which takes place in my 
children's literature course. After the disagreements have been aired, the class goes on 
to discuss what we see and what we know that allow reasonable people to make such 
divergent interpretations of the same texts and illustrations. We try to find evidence in 
the illustrations and the words of the books for all points of view, and we discuss the 
nature and validity of the evidence we find. But ultimately, we have to turn to an 
examination of ourselves and the knowledge we have about the artist/ authors, about 
the nature of picturebooks, and even knowledge about the society we live in. 

Lately, I've begun to realize how much these discussions with my students have 
affected my thinking about these two books. I still enjoy them for the same qualities 
that originally attracted me and led me to include The Three Robbers and Ben's 
Trumpet in my course. I also recognize that there may be unintentional racism m Ben's 
Trumpet, the result of a well-meaning white person's efforts. At the same time, I can 
still read Ben's Trumpet as the story of the unrecognized child artist who needs the 
assistance of an understanding adult. This is a universal theme and not uncommon in 
children's literature. In the case of The Three Robbers, I have to confess that while I 
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think the book is ? parable about the unexamined life, I can also understand how a 
casual reader could interpret it as teaching children that stealing has its rewards. I 
have to carefully point out to students that the robbers are not rewarded for stealing; 
they are rewarded for making reparations in the form of caring for orphans. I also tell 
my students that Tomi Ungerer would agree with Marshall McLuhan that art is 
subversive; Ungerer is deliberately subversive and provocative in many of his 
picturebooks so that his work needs to be examined thoughtfully for its messages. 

It has been my contention that the messages and values of picturebooks, novels, 
nonfiction, and all types of texts can be most thoroughly discovered through group book 
discussion that has some structure and a facilitating leader. My colleagues will discuss 
some specific situations in which book discussion is particularly valuable and helpful. 
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Beyond Reviews: 
Using Book Discussion as a 
Method for New Materials Selection 



Kathleen T. Homing 



In her opening statement Willett spoke of the importance of pooling our resources to 
make adequate evaluations of books. One of the resources she mentioned was 
professional reviews. I would imagine that all of us working in public libraries use 
reviews to some extent in new materials selection. If s one of the things we're taught to 
do in library school. Many libraries have the systematic use of reviews built right into 
their selection policies, as in instances where a libtary must have at least two 
favorable reviews on file for a particular book to justify its purchase. 

The extent to which we as public librarians rely on reviews varies. Out of necessity, 
many librarians must rely on reviews completely for new materials selection, as they 
have no access to an examination collection, no opportunity for first-hand evaluation of 
newly published books. Good professional reviews inform us. They tell us what a book 
is about, describe artwork, point out strengths and weaknesses, suggest age-levels, and 
occasionally recommend ways a book might be used, as in "great for reading aloud" or 
"easily accessible for homework assignments." But as Willett pointed out, even the best 
of reviews provide what is essentially a one-sided affair: you can't have an exchange 
with a book review. And as those of us who have had opportunities to use examination 
collections of new books know, there is simply no substitute for first-hand experience 
with a book. 

I have been in situations, as I imagine we all have, where I have only reviews to go 
by and no book. Sometimes all the reviewers agree that a particular book is wonderful. 
Sometimes they agree that a book is ordinary or mediocre or flawed. And sometimes, 
the reviewers disagree. I find all these situations frustrating, because there is no chance 
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for exchange, no chance to ask the reviewer questions, no chance to make my own 
observations and judgments. I end up, in this situation, making observations and 
judgments about the reviews rather than the book which is reviewed. Three reviews of 
a recently published book, A Balloon for Grandad, will serve as a case in point. The 
reviews from Booklist and Bulletin for the Center of Children's Books are very 
positive; the review from School Library Journal is quite negative. The first two 
reviews have stars— and I confess that stars weigh heavily in my own decision-making 
process. But how many of us have had the experience of knowing a book and seeing a 
starred review of it and thinking "That was starred?!" All of the reviews comment on 
the high quality of the artwork in A Balloon for Grandad but I find that I am troubled 
by the third review from School Library Journal which is so negative in its criticism of 
the story. And without having a chance to see the book myself and with the 
consideration that the reviewer for School Library Journal is the one reviewer 
represented here who is a practicing children's librarian, I ask myself: Is this another 
one of those picture books that dazzles you with its artwork but has very little to offer 
in terms of child appeal or story? You know the kind of book I'm referring to! 

As it happens, I have the book itself here and as you can see the artwork is 
beautiful. Nothing, not even Betsy Hearne's particularly eloquent description of the 
art, compares with actually seeing it for yourself. First-hand experience with the book 
allows me to make my own observations and judgments about it. I observe, for example, 
as none of the reviewers had pointed out, that in this family the father and the little 
boy are light-skinned and have brown hair and blue eyes, while the mother and 
grandfather are dark-skinned and have black (or white, in the case of the 
grandfather) hair and brown eyes. 

Those of us who have the opportunity to see the books first-hand in order to make 
selections of new materials for libraries are very lucky indeed. Working here at the 
CCBC over the past nine years, I have had the opportunity not only to discuss new 
books in a number of different contexts, but also to observe the selection processes of the 
many, man/ librarians who come here from all over the state to go beyond reviews, and 
take a first-hand look at the books they are considering for purchase. In speaking about 
her film What's a Good Book?, the Weston Woods film about library book selection, 
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filmmaker Joanna Foster once commented that she faced a real challenge in making this 
film, because most of what happens in book selection happens inside a person's head, 
and doesn't make for very dynamic film footage. This I would say is true of most of the 
book selection I have had the chance to observe. A common bit of scenery here at the 
CCBC is a librarian, seated at one of the book carrels out in our reading room, facing a 
long row of books with little 3 x 5 review cards sticking up out of each book. She grabs a 
book, pulls out the reviews and scans each one, then turns to the book. If if s a picture 
book or a short work of nonfiction, she may read the entire thing. If if s a longer work, 
shell read an excerpt, check out the front matter and the back matter, make a few 
pencil marks on one of her 3 x 5 cards, shove the book aside and move on to the next one. 
All of this may happen in a matter of minutes, and in some instances, seconds. I can't see 
all of the things going on in her head, but as a public librarian myself who occasionally 
does this sort of book selection, I know that she's looking at format and type size to 
make judgments about accessibility and age level. She's thinking about the library 
collection she works with, making judgments about how this book would fit in and 
where. She's thinking about the community she serves and considering using this book 
in a preschool story hour or suggesting it in her readers' advisory work. And, she's 
thinking, overall, about her book budget. 

One thing I observe quite frequently when there is a group here from one library 
system, is that process as I've described i' above with one addition. At the end of the 
process, instead of shoving the book to one side, she'll hand it to a colleague and say, 
"Here. Tell me what you think of this." This is an opening for a book discussion-not a 
formal structured discussion, but a book discussion nonetheless. There are times when a 
formal structured discussion is not necessary or possible. Librarians who work together 
and have a common base of experience, in the sense that they serve the same 
community, work in the same library system, often develop a type of shorthand which 
allows for this type of discussion. It requires good communication skills, trust, 
confidence, and sometimes includes a recognition of special skills various colleagues 
bring to the job. For example, if I have some questions about the effectiveness of a 
particular board book, I refer them to my colleague Rose Ann Ableidinger, who has had 
a great deal of experience doing toddler story hours and has a good understanding of 
how babies respond to literature. This type of discussion is brief and informal, but I find 
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that 1 learn a lot from Rose Ann and that by asking her questions, I can hone my own 
evaluative skills when it comes to board books* 

Some of the shorthand discussions that go on in the CCBC reading room get pretty 
lively, and occasionally spill over onto the CCBC staff. If one of us happens to be 
walking by on our way from one place to another, we may be handed a book, with the 
question "Have you read this? What do you think of it?" And the discussion goes on. 
Because the CCBC staff doesn't have the same base of common experience as the 
librarians here for book selection, ihe discussion might take a little longer. We don't 
have the shorthand. Ginny Moore Kruse might point out something in her comments 
about the book that the librarian hadn't noticed or thought of or vice-versa, This sort 
of thing goes on here nearly every day, 

I have also, albeit less frequently, had the occasion to observe some more structured 
book discussion when librarians have been here to evaluate books. What is typical in 
these situations is for one librarian in a group to give an oral review oi a book she has 
read, after which there is an opportunity for her colleagues to ask questions or make 
comments. In addition to discussing the merits of a particular book, these discussions 
frequently include a discussion of whether or not the title should be purchased, and in 
what quantity. This is a type of book selection that you can see. What Joanne Foster 
described as happening in the head is out in the open. 

What I have sometimes observed at these type of book discussions is one person, the 
only person who has read the book, holding it up and then tearing it to shreds. This, I 
have observed, closes the door on book discussion. Who wants to waste time asking 
questions about a book that has uo merit? 

Of course, the way to avoid this sort of situation is to have more than one reader for 
a book, Then there can be dialogue and a discussion, the sort that Willett described so 
well in her opening speech. This is an ideal, very hard to reach, because it requires a 
commitment of time and staff, But it builds on skills we already have, And the rewards 
of a formal structured book discussion as a means of evaluating new materials are 
boundless, personally and professionally. Book discussion helps to hone the critical 
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skills important in evaluation. I know that this is true in the continuing development of 
my own critical skills and I have seen it happen in colleagues with whom I have 
discussed books over a period of months or years. Sharper critical abilities allow us to 
be more selective in ?ur evaluation of materials. We know what we're buying and 
w hy— what we're not buying and why not. 

Book discussion allows for the ultimate pooling or resources. Willett used the term 
"toolkit" in her speech. I'd like to talk for a moment about the librarian's 
toolkit-what we bring to our reading in addition to our own personal experiences, 
knowledge, and abilities. We bring our professional experiences, knowledge and 
abilities. These would include our experiences with a particular community we serve, 
our knowledge of the existing library collection, and our ability to translate this 
knowledge and experience into library services for children. When I was describing 
earlier that process of the lone librarian sitting at a carrel selecting books, what I was 
observing was the librarian delving into her professional toolkit to evaluate materials. 
But I couldn't see this toolkit. Book discussion makes it visible and not only makes it 
visible, but provides an opportunity to add to it. I'd like to give an example of how this 
can work. 

A few years ago I participated in a book discussion in which one of the titles under 
consideration was The Burning Questions of Bingo Brownby Betsy Byars. Everyone had 
favorable comments to make about the book, including me. There was one person in the 
group, however, who pointed out that she was troubled by one aspect of Bingo's 
behavior. If you've read the book, you may remember that, throughout it, Bingo feels 
he is under so much stress that he has difficulty sleeping, so he is constantly popping 
junior aspirins to help himself to go sleep. If s a comic device, as Byars intended it to be. 
I had thought it funny when I read it, as did a number of other people at the discussion, 
if they thought much about it at all. The one librarian I have mentioned, however, 
found it disturbing, not only that such behavior went on throughout the book without 
ever being challenged, but that we as readers were lulled into believing that popping 
pills was funny. Well, I think we all sympathized with this librarian's concerns, and 
some of us were, I think, slightly disturbed that we hadn't had the same concern 
ourselves. If we were relying completely on reviews in considering The Burning 
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Questions of Bingo Brown, the pill-popping issue would never have come up. If we were 
relying on only my reading of the book, it wouldn't have come up either. I missed it 
Now imagine the librarian who brought it up in the situation where she was the sole 
reader reviewing it for a group, in the type of one reader book discussions I described 
earlier. She might have chosen to focus on this issue and make it central to the 
discussion, tearing the book to shreds. There would not have been the give-and-take, or 
the chance for positive responses to the book which a discussion with more than one 
reader allows for. 

Of course, we bought The Burning Questions of Bingo Brown. It received excellent 
reviews; it went on to be a Booklist Editor's Choice and a School library Journal Best 
Book of the Year. And, besides, Betsy Byars is an extremely popular author. Her books 
are in high demand. So, of course, there was no question but that we'd buy The Burning 
Questions of Bingo Brown. We bought it, however, with the knowledge of the 
pill-popping scenes and, while this knowledge did not influence our decision about 
putting it on the shelves, it does influence our decision about featuring it in 
recommended book lists, bibliographies and displays. And I find that I myself hesitate 
to single it out for a recommendation for a good, funny book. 

Selection doesn't stop with purchase. We continue to select books every day. We 
select books for bibliographies and displays. We select books for programs, such as 
preschool story hours and book talks. We select books for individual children as we 
provide reference and reader's advisory services. Using book discussion as some aspect 
of the selection process can help to put us a little closer to our goal of providing the 
right book for the right child at the right time. 
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We most often think of book discussion in the context of evaluation at the time it is 
considered for initial purchase. Book discussion can also be a useful <ool at the time of a 
complaint. 

How is book discussion used at the time of a complaint different from discussion for 
purchase or programming? There are five differences that will be focused on: one 
difference has to do with the players; another difference has to do with the focus of 
the book; another difference has to do with the background or context in which the 
discussion occurs; a fourth difference is the opportunity for discussion; and finally the 
fifth concerns at what stage such a discussion occurs. 

First, the players. The discussion we might have about a book that is being 
reconsidered can include several groups of people. The first is the group that we 
commonly think of, that is, children's librarians talking about children's books or other 
materials. In a reconsideration discussion we might have other players. These players 
can include other librarians who are involved at the early stages of discussion, such as 
directors, or reference or adult services librarians as well as members of the public 
library staff who do not have a background in children's literature but who have a 
responsibility for selection. We also have the players who are involved in the context 
of the more formal stages of the decision, particularly members of the public library 
board who may participate on committees or in the final review process. These persons 
certainly have an interest in library materials, but they do not usually have formal 
library training in children's services or literature. 

The second difference is the focus on the book or material for reconsideration. The 
book has been read or reread in most instances with the knowledge of what is deemed 
"wrong" with the book, rather than approaching it from the viewpoint of what is 
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positive or appreciated about the book. The openness that one has coming into a book 
discussion knowing that you will hear other viewpoints that will help you learn or 
grow in your understanding about the book is not likely to be present in this type of 
discussion, The focus also relates to how the participants will approach the 
reconsideration. In many instances the participants will have their minds already 
made up. I am using the term participant here to include the person who has 
complained about the material; that person has his or her mind made up, and you as 
the children's librarian probably have your mind made up, too. So we have people 
involved in the situation who have their minds made up and we are approaching the 
discussion with something already pointed out about this book that is perceived to be 
wrong. We also have a focus here— we think more particularly in terms of the child's 
toolkit referred to earlier by Willett. We think in terms of the child as one who is 
growing, developing, and absorbing information. Adults have concerns about what is 
presented to children, and we think about all of these issues as we approach the 
discussion, 

The next difference that distinguishes book discussion during a complaint from that 
of purchase or programming is the background or context in which the book is discussed. 
We are looking at it more specifically in terms of the selection or collection . 
development policy of the public library or public library system. We are thinking 
about the specific criteria that have been established, and are looking to see if those 
criteria were used in the selection of the material. So when you are reconsidering, the 
selection or collection development policies and the criteria in those policies will 
assume a prominence. While you know that the selection policy is there in the 
background when you select materials, in a case of reconsideration you are looking at it 
closely, looking at the book closely, and comparing the two because you are making sure 
the criteria were followed. The examination of the material will also reflect previous 
discussion that has occurred, if that has been part of the selection process. This can 
serve as a reminder, as a reaffirmation of what was most appreciated about the 
material. 
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It is likely to be helpful to be reminded of the oral discussion involved in the 
selection of the material. At the time of a complaint we also rely on reviews more than 
we would in open discussion. The person or persons who are going to make the final 
decision on whether the material is going to be retained will want to be aware of 
reviews. Depending on the process used in selection, you may be using reviews 
developed locally or in the system, but you will also want to use national and regional 
reviews about the material itself. The selection policy and reviews will therefore take 
on an additional prominence. The final context in which I would look at how a book is 
discussed during a complaint process is a context in which the book itself is not discussed 
and is what 1 would call the intellectual freedom context. That is to say, in some areas a 
book that is questioned is looked at in terms of intellectual freedom, in terms of access, 
in terms of the importance of having varying viewpoints. When we look at it in terms of 
intellectual freedom, sometimes the book itself is not discussed or is not looked at, but 
the whole process is looked at in terms of whether material should be accessible, and 
the mission of the public library. Some feel it is not necessary to go into the specifics 
about something that has been questioned but to rely on the principles of intellectual 
freedom to defend having material in the collection. The background or context in 
which the book is discussed may therefore include: the policy, reviews, previous 
discussions, and what the policy says about how material will be defended and 
discussed, and the intellectual freedom focus which is more absolutist and which may 
not permit, or encourage, discussion of the questioned material. 

The opportunity for book discussion during a complaint will be determined by the 
procedures that have been established in the collection development policy. There are 
many varieties. Here are a few: the request for reconsideration may go directly to the 
library board and be considered at that level; the request may go to a committee of 
children's librarians and that committee would make recommendations that would go 
to the children's coordinator and on to the director; the request may go to a committee of 
librarians which would include but not be limited to children's librarians, and would be 
chaired by the director; the request may go to the selection committee for children's 
materials or to the director and action is taken following review by either the selection 
committee or the director; request may go to a group of children's librarians, who do not 
formally meet as a committee, but instead review the material separately, come to a 
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conclusion/ and submit a personal recommendation in writing to the designated person. 
There are certainly subsets and many variations of the above examples. Opportunity 
for book discussion therefore depends greatly on what procedures have been set up for 
reviewing the material when there has been a complaint. What most of these practices 
have in common is that the complainant is not present to hear or to be involved in the 
initial discussions and in some cases the discussion is going to end there* In other cases 
the complainant comes into the picture only at the point at which a decision is 
appealed to the public library board. 

A final difference in book discussion at the time of a complaint is the stage at 
which a discussion occurs. We have what I would call initial stages when a 
reconsideration is considered by a group of librarians in a meeting setting and that is one 
of the stages where open discussion can occur. We also have the more formal stage 
where the reconsideration would be considered at a meeting of the library board. There 
are some findings in communications research that would seem to be appropriate here. 
There is much in this field that we would find helpful in thinking about discussion and 
group process that we as children's librarians will want to spend more time learning 
about because it relates to how people communicate and how we are able to share. The 
two findings I'll share today relate to interpersonal discussion and interaction. 
Interpersonal discussion over time tends to increase accuracy more than agreement 
Discussants don't end up seeing eye to eye after debating, but they have a better 
understanding of how people think. Another finding in communications research that I 
think you will find particularly interesting is in terms of reconsideration, the person 
who has complained, the point at which that person is involved, and the manner in 
which the complainant is involved. In an interpersonal encounter (such as a 
conversation or a discussion), people react to one another as unique individuals. They 
can sit and talk with an individual. In non-interpersonal situations (such as a 
reconsideration hearing), people respond to others more in terms of stereotypical roles. 
So that if the person who is complaining becomes involved primarily at the formal 
meeting stage, the person is going to see the director or the children's librarian, not the 
people who have these roles, and this makes a difference in communication. 
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There are some questions we need to think about as we look at reconsideration and 
the whole area of book discussion: 

1 . Should the complainant have an opportunity to listen to a book discussion of 
the book in question, or even to participate in such a discussion? You have heard 
about how much individuals grow and how we have an opportunity to look at 
the whole. We get all these viewpoints from all these directions. Is this 
something that the person who has complained about the book should have the 
opportunity to hear or to participate in? If your answer is yes, then at what 
point should the complainant participate in or listen to a discussion of the book 
in question? 

2. Should librarians who are not children's librarians who are involved in the 

consideration of the title participate in a book discussion of the tide thaf s 
been questioned? 

3. How might the atmosphere of a formal reconsideration meeting change if a 
book discussion of the type we've been learning about sets the stage for a 
discussion about the material that has been questioned? 

4. Is the retention of the material all that matters? Might there be other 
"victories" if book discussion sets the stage for the reconsideration of 
materials? 
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Discussion for Awards and Distinctions 



Eliza T.Dresang 



I will address the difference between an award and a distinction before delineating 
the characteristics of evaluative discussion when awards and distinctions are the 
expected end product. 

Webster defines award as: 

a judgment or final decision; something that is conferred or bestowed 
especially on the basis of merit and distinction as: discrimination or 
differentiation; a special honor or recognition 

The words "judgment or final decision" in the definition of an award are significant 
and it is this concept in which an award differs from a distinction. An award honors a 
single book, e. g. the Newbery Award, the Coretta Scott King Award, while a 
distinct* * is given a book which is part of a group of books or a body of works having 
something in common, e.g. Notable Children's Books, Best Books for Young Adults. 

The above definitions are important because, just as discussion of books considered 
for an award or distinction differs somewhat from other evaluative discussion, 
discussion of a book considered for an award differs somewhat from discussion of a book 
considered for a distinction. Evaluative discussion for books considered for either 
awards or distinctions differs from other types of book discussion in both manner and in 
degree. 

The manner in which these discussions take place results from the framework 
placed around the discussion. This framework comes from the special criteria which 
must be kept in mind throughout the discussion. Just as general discussion rules are laid 
out for participants at the beginning in evaluation of books for any purpose, the specific 
criteria for an award or distinction must be communicated to participants and adhered 
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to assiduously throughout both personal and group evaluation if the end result is to 
meet the intended purpose. Preparation for Ms type of discussion requires more time in 
order to place some parameters on the personal views brought to any discussion. 

For example, prior to the first meeting of the Newbery Committee upon which I 
served, I engaged in a brief conversation with one of the committee members. "I am 
determined/' she said/'that this year's winner will be accessible to elementary school 
children. We have too many Newbery winners for the older reader." While this is most 
likely the secret hope of many of us each year, one can comb the criteria for the 
Newbery Award and find nothing which sets this as a justifiable guideline. 

In discussion of books for awards and distinctions, it is t iore important than perhaps 
for any other type of discussion to examine one's own agendas and find whether they do 
or do not fit the task at hand, to consider consciously what Willett has defined in her 
second observation about book discussion as "finding oneself in one's reading." By its 
nature, discussion for distinction has more carefully defined restrictions than discussion 
of any other type. A value of the group discussion is that when one does tend to stray 
from remarks which are helpful to the end goal, other group members readily guide the 
discussion back to its intended purpose. This generally does not present a problem once a 
group has reached a common understanding of the criteria which must guide the 
discussion. 

Sometimes criteria for an award or distinction present a particular challenge to a 
committee because of their very nature. They may call for specialized knowledge 
which, if not possessed by the committee itself, must \r obtained from outside experts 
and brought to the committee. Here "right" cr "wrong" may exist in an evaluation. For 
example, the translated book chosen for the Batchelder award, an award which is 
presented to a publisher rather than to a book itself, must retain the true 
"substance... and flavor of th^ original work and should retain the viewpoint of the 
author" and "should not be unduly Americanized." On the other hand, another criterion 
reads that "in all cases, the committee must consider the book's manner of presentation 
for and the potential appeal t j a child (read 'American child') audience." This brings a 
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focus to a distinction which might not have occurred had not the book been eligible 
an honor. 



One problem which faced this year's Batchelder Committee included whether 
kibbutz life was authentically portrayed in The Boy From Over There, (Houghton, 
1988), and, if so, was enough detail given to make it understandable to an American 
child! As Willett said, adults can only guess at the answer to this question, but that 
estimating most certainly affects the manner of discussion. To include this book in a 
general collection of books for children would involve such a discussion but the 
resolution might be unimportant because at least a number of children would bring the 
background to understand the book, and the focus of discussion could be on literary merit, 
etc. But to single this book out for an award, one must be far more certain that the 
culture portrayed is intelligible. A different problem arose from the consideration of 
Faithful Elephants, (Houghton, 1988), a story of the extermination of elephants in the 
Tokyo Zoo during World War II, in order to prevent their escape into the city should the 
zoo be bombed. The nature of the slow and painful death of the animals and the 
decision to kill these animals in the first place which is not put into context because of 
the brevity of text, was a cultural element which the committee had to discuss. Again, 
such a discussion becomes xar more important when a book is to be considered for an 
award. The style in Crutches (Lothrop, 1988), the book whose publisher was awarded 
the Batchelder this year, raised yet another question. Was the style that of the author 
or translator? Here at the CCBC we were fortunate to have a woman in one discussion of 
this book who was quite familiar with postwar German literature and could address 
this issue of authenticity. 

/ Discussion for notable or best requires yet a different kind of mind set. 

Understanding what "worthy of note" means affects the manner in which the books 
under consideration will be discussed. Here we have "acceptance by children" rather 
than "appeal to a child audience" of the Newbery criteria. How does this alter the 
content of the discussion? The Laura Ingalls Wilder Award requires that the author 
have established a new type or kind of book or new trends in books available for 
children. Elizabeth George Speare, the most recently honored author for the Wilder 
Award, does not have a large body of work; yet the committee and all of you who voted 
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obviously believed that her four books established a new kind of trend in children's 
literature. Discussion of just what this was in evaluating these books was an important 
part or that evaluation. Once again th # . criterion will affect the manner in which the 
book is discussed. 

Another important factor in the manner in which these books are discussed in the 
group process. I had the good fortune of being trained in a media evaluation institute 
funded by ALA for all award committee members prior to my Newbery Award 
experience. I went to this workshop expecting to hear something perhaps along the line 
of what I have just delineated. Instead, I spent two days without one mention of media, 
criteria, etc. The entire workshop was on group process and on the roles and 
responsibilities of members of a group attempting to reach a decision. The tendency of 
some to dominate, others to get off track, and the necessity of peer group monitoring was 
eye-opening. The manner in which one must operate in an award /distinction committee 
cannot be casual, and group cohesi veness becomes paramount. 

Holding mock discussions which lead to local determination of awards or 
distinctions has great merit in the evaluative process in that participants come to 
understand the kinds of constraints and opportunities placed on such a committee. The 
influence of all the characteristics Willett mentioned, the limitations of readers, the 
bringing of one's self to the discussion, and the scrutiny of what the author has hir; or 
herself brought are all seen in a different light when the end result of the discussion 
must be the selection of one book or some books over others with justification for the 
results tied by necessity to the criteria. Issues which may cause controversy should be 
dealt with by these mock groups and outside opinion sought when necessary in 
preparation for the discussion. 

Manner of discussion influenced by the presence of the need for an end product 
reflecting specific criteria is one way in which evaluative discussions differ from 
others. The other difference is degree or depth of analysis which can and will be 
applied to any one book. Although the manner of discussion can be fairly successfully 
simulated in mock discussions, the depth of analysis is far harder to experience in a less 
than "real 7 situation. The intensity and depth of discussion come partly from the sense 
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of responsibility borne by committee members. Most committee members, preceding a 
discussion at ALA, have engaged in numerous discussions with colleagues and children. 
The level of preparation one brings to an award or distinction evaluative discussion 
affects the depth to which that discussion may go. In the Notable Books Committees 
which have a relatively short time to devote to each book, discussion starts far beyond 
where casual wscussion begins. In those committees considering awards rather than 
distinctions, the depth of discussion is even more apparent. The collective thought 
process does indeed have a sum that is greater than its parts and insights that simply 
are not possible in an initial discussion of a book emerge. 

To refer once again to the Batchelder Committee, this committee has a luxury 
which even the Newbery/Caldecott committee does not enjoy-fewer books and so a 
greater amount of time which can be devoted to the discussion of each book. The third 
or fourth hour we spent with each book brought more and more insights and 
understandings. We clarified more and more our judgments. Much of this discussion took 
place without reference to the criteria, and the ultimate decision rested upon returning 
to them, placing what we had learned against them. Even so, when I returned from 
ALA, following a year of many discussions, a student in the young adult literature class 
I taught this spring gave me yet another insight into Crutches that no one on the 
committee had grasped. The point of this is that the degree of discussion is deeper but 
never exhaustive in an award committee-this ties to WUletf s suggestion that 
continuous revelation about award winning books is necessary to, in the end, determine 
their true merit. But award committees can come most closer to this than can those 
engaged in more casual, less intensive discussions. 

Dicey's Song (Atheneum, 1982) is a book with which I am intimately familiar. It 
was the book selected for the Newbery Award the year I was on the committee. 
Following that I wrote an article for School Library Journal (Nov. 1983) scrutinizing it. 
In my initial discussion of this book, I did not fully recognize or appreciate the lyric 
quality of the text, c song in itself, nor did I understand how Voigt achieved it. I did not 
appreciate the many symbols which make a cohesive work. I did not understand Voigf s 
subtle method of characterization. But the depth with which this book was discussed 
led to insights that simply could not come in any other way. Once again, those insights 
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were not complete as the book received its award (they are still coming) but the 
understanding reached far beyond casual scrutiny. 

Discussion for awards and distinctions, therefore, differs in manner and degree, but 
not in basic purpose or structure from any evaluative experience. 
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Guidelines for Book Discussions 



Ginny Moore Kruse and Kathleen T. Horning 

©1989 Ginny Moore Kruse and Kathleen T. Horning. 

ALL PERSPECTIVES AND VOCABULARIES ARE CORRECT. THERE IS NO 
"RIGHT ANSWER OR SINGLE CORRECT RESPONSE. 

1) Listen openly to what is said, rather than who says it. 

2) Respond to the comments of others, rather than merely waiting for 
an opportunity to share your comments. 

3) Talk with each other, rather than to the discussion facilitator. 

4) Comment to the group as a whole, rather than to someone seated 
near you. 

LOOK AT EACH BOOK FOR WHAT ITIS, RATHER THAN WHAT IT IS NOT. 

1) Make positive comments first. Try to express what you liked about 
the book and why. (e.g. "I like the illustrations because....") 

2) After everyone has had the opportunity to say wrta* *hey liked about 
the book, you may talk about difficulties you had • -th a particular 
aspect of the book. Try to express difficulties a ' ons, rattier 
than declarative judgments on the book as a \ .e.g. 'Would 
Max's dinner really still have been warm?" raw . than 'That 
would never happen.") 

3) Avoid recappLig the story or booktalking the book. There is not 
time for a summary. 

4) Refrain from relating personal anecdotes. The discussion must 
focus on the book at hand. 

5) Try to compare the book with others on the discussion list, rather 
than other books by the same author or other books in your 
experience. 
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Commentary on an Approach to Discussion: 
The CCBC Guidelines for Book Discussion 



Holly Willett 



It's 3:30 on a Friday afternoon. In the conference room at the Cooperative 
Children's Book Center (CCBC) people are gathering for the monthly discussion of 
newly published cluldren's and young adult books. Most of them will have read all of 
the nine or ten books chosen by Ginny Moore Kruse and Kathleen Horning three or four 
weeks before, and all of them come voluntarily to discuss the books. Some of them may 
have driven for two hours or more to participate in the discussion. The only requirement 
is that participants have read some of the books to be discussed-no spectators 
allowed! Today the group may include a children's librarian, an elementary school 
teacher, a writer or illustrator, a school library media specialist, a preschool teacher, 
university students from various disciplines (both undergraduate and graduate 
students), university faculty members, and a staff writer for the local newspaper. Some 
members of the group may have multiple connections with children's books, such as the 
teacher who also works in a children's bookstore or the librarian who is also a parent. 
For the next two hours, this varied group will discuss the books, following the 
guidelines written by Ginny Moore Kruse and Katy Horning and reproduced on the 
preceding page. 

When Kruse began the monthly book discussions in 1979 with the assistance of 
Susan C. Griffith, they did not have guidelines. While the discussions were 
wide-ranging and intense, Griffith and Kruse realized that it was very easy to find 
fault with a book and spot its weaknesses. Sometimes it was difficult to remember a 
book's strengths when people started with a negative approach. It was possible to 
leave a discussion feeling that children's books were not being done very well. Books 
were not being appreciated for their positive attributes, and perhaps participants were 
not being led to expand their abilities to evaluate books. Hence, unwritten guidelines 
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were developed. In 1985, Kruse and Katy Horning wrote out the guidelines. Since tb*n, 
experience with the guidelines has led to some modifications and fine tuning. 

The three basic principles of the CCBC book discussion guidelines can be applied to 
any discussion for the purpose of evaluation: (1) Begin with the positive, (2) welcome 
all views, however expressed, and, (3) keep to the matter at hand. As Kruse and 
Griffith found, it is easy and tempting to launch an attack without stopping to consider 
the strengths of a book. The same is true when one is evaluating a program, a procedure, 
a building, etc.; the positive aspects can be forgotten or ignored. To use Lynn 
MacDonald's woras, we have a "deficit model" of how to evaluate. A better balance is 
struck if the positive is voiced first. It may take some looking to find it, but most books 
and situations have something to be said in their favoi. Interestingly, what one person 
views as a positive, another may see as a negative. Persons who disagree with someone 
else's positive statement should wait until all positive ideas are expressed before 
making t\eir negative comments. In discussions where the relationships among the 
participants involve a hierarchy, this issue is especially sensitive, but in .ny group, it 
is important that ideas be criticized, not the person who holds them. Hence, the second 
principle, welcome all views. 

In addition to possible disagreements, there are many occasions when discussion 
may be used for evaluation with as disparate a group as that which meets to discuss 
new books at the CCBC. Clerical staff, parents, children, representatives of community 
agencies, teachers, all may have valuable insights to contribute, but their vocabularies 
and agendas may not be those of children's librarians. The guidelines suggest listening 
to the content of what is said, responding to group members, and commenting to the 
entire group. A capable facilitator assists people to appreciate and understand all 
points of view, acting as a translator and group memory. It is easy to think that there 
are just two points of view for any issue, but in reality there are many more than two. 
Because of the multiplicity of opinions, the third principle becomes crucial. 

Keeping to the matter at hand means using the available time well, which is why 
there are no summaries or booktalks as part of the discussion at the CCBC and why 
personal anecdotes or comparisons to books not on the list are discouraged. The 



Willett SCHOOL OF LIBRARY AND INFORMATION STUDIES 

270 



expectation is that as many people as possible have read the books and that reading is 
the shared experience that everyone has, whereas personal stories and comparisons 
with other books or events may not be meaningful to all participants. The same should 
be true in evaluating other kinds of phenomena-participants should have a shared 
basis for discussion. Keeping to the matter at hand also means that a complete picture 
is desired, that the book or the situation should be seen as a whole. And in order to do 
this, the principles of being sure that the positive is stated and welcoming all views 
help to achieve the goal of keeping to the matter at hand. 

Does the CCBC method meet its objective of helping people appreciate children's 
and YA books? We decided to test it in a very simple pretest-posttest design without a 
control group. As preparation for the Institute on Evaluation Strategies and Techniques 
for PubUc Library Children's Services, participants received a list of five books to read 
before arriving in Madison; the books were to be discussed in small groups during the 
book evaluation session, following the CCBC guidelines. After reading the books and 
before the discussion, participants filled out a one-page questionnaire about their 
responses, which included such questions as 'To whom would you recommend this book?" 
and "What is your overall rating of the book?" After the discussion, the participants 
again filled out the questionnaire for each title. 

In general, after the discussion participants rated the books higher and indicated 
more uses. That is, there were fewer ratings of Poor or Marginal and more ratings of 
Very Good and Excellent. More of the participants said they would recommend 
purchasing the books after the discussion than before. Although chi-square tests of a 
number of variables revealed only modest changes at the 20 and 30 level-which are 
not statistically significant-t tests of independent means on the overall ratings found 
that two of the five books were rated significantly higher after discussion, at the .02 
level and the .05 level, respectively. The other books did not show significant gains in 
ratings, but only one of the five books showed slightly more negative responses after 
discussion than before. One would probably not expect to see great changes after a few 
minutes of discussion. The testing and its results are certainly a limited, though 
intriguing glimpse at the possible outcomes of discussion. 



WUlett 

UNWERSHY OF WISCONSIN— MADISON 

271 



9 

ERIC 



The CCBC monthly book discussions are held for the purpose of introducing new 
children's and YA books to those interested and to encourage the discussion of materials 
created for the young. No awards are given, though special book discussions are held in 
December to discuss books which may be on the lists of the Newbery, Caldecott, 
Batchelder, and Coretta Scott King Award committees. The award discussions at the 
CCBC are carried out using the CCBC guidelines and the balloting procedures of the 
Association for Library Service to Children and are intended to provide opportunities 
to improve discussion and evaluation skills and to understand the award 
decision-making process. In an informal atmosphere, such as the monthly book 
discussions at the CCBC, there is no need to arrive at consensus. In situations where an 
agreement or a decision is desired, some mechanism must be built into the discussion 
process for arriving at a group result. Simple balloting — one participant, one vote — is 
the usual method. In other situations, more elaborate procedures are necessary, such as 
the weighted balloting that is done in the book award committees sponsored by the 
Association for Library Service to Children (Peltola, 1979). Weighted ballots require 
participants to rank the possible choices and result in a possibly more refined 
distinction among the alternatives. The Newbery and Caldecott Award procedures also 
stipulate that at least eight of the fifteer committee members must rank a book first in 
order for that book to be given the award. 

Although we have examined book discussion in some detail and have outlined 
methods and contexts for book discussion, evaluating books is not the only purpose for 
which structured discussion is appropriate. For the sake of a balanced perspective and 
a democratic process, the discussion method outlined here may be used to evaluate 
programs, policies, procedures, facilities, services, possibly even personnel with the 
"group" limited to the supervisor and the subordinate. Imagine sitting down with a 
group of colleagues and relevant others to discuss the summer reading program. Imagine 
beginning the discussion with all the things people liked about the program, instead of 
starting with the areas believed to be weak. It is both more reassuring and more 
realistic to give staff some positive support so that any weaknesses can be dealt with 
positively. 
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The CCBC guidelines were designed and developed through experience in small 
group discussions centering on books. I have tried to suggest how the guidelines function 
not only in book discussion but how they might apply in other contexts as well. Often 
after a CCBC book discussion meeting, I find that my involvement with those 
particular books has only just begun, that I will go on to read reviews of the books or 
continue the discussion with students or colleagues. Just as one discussion is not the final 
word on any book, so structured discussion of most things is part of a continuing process of 
evaluation. Long after a building is built or a program is completed or a procedure is 
implemented, those involved will be debating the merits of the building, the program, 
and the procedure. Like other forms of evaluation, discussion is meant "to improve, not 
prove." 
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V. Feedback into Planning 



Introduction 



In the long run, the purpose of evaluation is to provide additional information so 
that decision making about library activities may be improved. However, evaluation 
results are not automatically included as information for decision making. The 
evaluator must convince decision makers that the information obtained from 
evaluations is both accurate and relevant to an important aspect of the library's 
activities. In libraries where little formal evaluation has been undertaken in the 
past, it may take some time to have information derived from evaluations used in the 
decision-making process. Even where the environment is conducive to accepting 
evaluation results, political or financial concerns may override what appears to be 
clear evidence to support specific decisions. 

This section includes two pieces. The first is an edited transcript of a speech by 
Jane Goodwin, Coordinator for Evaluation and Information Development. Fairfax 
(Virginia) County Public Library. Her message states that in order to ensure that 
evaluation studies contribute to managerial decmon making evaluatort must 
understand how the questions asked, data collated, and ruommendatunu made fit 
into the library's decision making framework Toe role of timely, accurate 
communication ts critical in ttduang perceived threat while alsc supporting ^tum 
derived from evaluation results Tht tecond pwct u the tetimsual 'Kepemng the 
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Results of Library Research" by W. Boyd Rayward. Vie will lei this treat speak for 
itself. 

Although evaluation is a broad and sometimes quite intricate responsibility, we 
believe we have presented you with a context and some techniques and procedures that 
will allow you to start an evaluation program for your children's services. We 
therefore ask you >o go forth and evaluate. 



SCHOOL OP LIBRARY AND 
INFORMATION STUDIES 



The Politics of Evaluation 



Jane Goodwin 



As Coordinator of Evaluation and Information Development in the Fairfax County 
Public Library, I seem to hold a unique position in Libraryworld. The Office of 
Evaluation and Information Development (OEID) was created in 1982 by director Sam 
Clay, soon after his coming to Fairfax County, to provide information needed to guide 
decisionmaking. Fairfax County is in Northern Virginia near Washington D.C Steady 
growth in the County over the last two decades has resulted in a number of commercial 
centers with residential areas in a cluster development pattern. The 400 square mile 
county area is served by 22 branches including six regional libraries, nine community 
branches and seven mini-libraries. The System has no central library. The County's 
population is currently about 750,000, with over 930,000 expected by the year 2000. 
Fairfax County is a prosperous area and has, on the whole, well-educated residents 
who are demanding library users. It seems that everyone has gifted and talented 
children! 

OEID is staffed by two persons. In addition to the Coordinator there is a full time 
assistant who is also a librarian with experience in children's services, mini-branch 
management, information services, and she is an expert online searcher. The Office has 
four formal responsibilities and many additional tasks that are covered by that part of 
our job descriptions that say "other dutfcs as required." Officially we are responsibUe 
for all evaluative studies conducted within the library system whether they are 
systemwide in scope, branch, or department specific. I have said to my assistant that 
we are going to know when we have really been successful in this job when we eliminate 
the need for our positions; when everyone thinks of evaluation when they are planning 
a library service or service enhancement. She has assured me that no one is ever going 
to want to deal with all the piles of data that come into the office at the end of a large 
survey. I often plead with my colleagues to remember, when planning an evaluation, to 
call us early in the project. We will assist in designing the evaluation component from 
the beginning of the project. We are not cheerful when called after data has been 
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collected and no plans for analysis are in place. OEID also serves as a clearinghouse. 
As much as possible we want to evaluate a service once, not over and j ver again at 
different sites. By maintaining an awareness of what evaluation efforts are in progress 
throughout the Library, we can reduce duplicate efforts. 

A second responsibility assigned OEID is to conduct feasibility studies for new 
services that the Library is considering. These studies may be requested by the 
Director, the Library Board or other Library Administration members. Usually these 
papers are prepared with a staff task force. 

A third responsibility is the oversight of online information services. FCPL offers 
access to DiALOG, Vu/Text and BRS at the regional libraries All regional information 
service?* librarians are trained online searchers. As a fourth responsibility we also 
explore and recommend new technologies for the delivery of information services. At 
the present time we are looking at a variety of CD-ROM products to be on regional 
information desk terminals. 

The last two responsibilities may seem a little out of place considering the first two 
tasks but OEID is in the Technical Operations Division. The Division operates with 
thin staffing levels and these responsibilities are well suited to our interests and 
experience. 

The topic I have been asked to address is the Politics of Evaluation/' The 
abstract in the Institute notebook stresses the importance of understanding how 
evaluation contributes to managerial decision making. I also want to stretch beyond 
that statement and *vork with a broader concept. For this presentation let us accept the 
definition that politics is the science of understanding power as it relates to the 
organization. This power includes decision making but it also includes the ability to 
implement a decision. Usually the authority to make decisions is formally placed in 
the hands of one or a few persons. Whether the decision is carried out or implemented 
successfully is really the work of many. There is assigned recognized power and there is 
unofficial or personal power. When planning and dealing with evaluation it is c itical 
to success that we develop an understanding of formal and informal power, and tne 
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power holders as related to the issue we are trying to evaluate. That b .at I mean by 
the phrase "the politics of evaluation"-** understanding and use of formal and 
informal power in the evaluation process. 

I have identified five groups or entities that I think it would be wise to remember 
when planning an evaluation in a library setting. First there is the community, the 
environment that supports the library-. Second is library management, the 
administration, usually the formal authority for decision making. Third are your 
colleagues, department heads and managers of other subunits in the library. Fourth is 
the staff and finally, the fifth powerholding group is the public. The public for us has 
always meant library users. At FCPL we have not conducted evaluation efforts outside 
the library although we make extensive use of data compiled by other County agencies. 

We have heard it a hundred times this week that evaluation is much more than 
measurement. It is a process of checking to see how close we are to a goal and 
recommending decision points that move the organization closer to the goal. The 
process involves at least these steps: you must clarify goals and objectives. You have to 
identify the question that needs to be answered and the piecos of information that are 
needed to answer that question. You must identify the audience for the evaluation. 
Who is going to use the information you collect? Another step is conducting the study, 
the measurement activity that collects the data. The next step is very important; you 
must convert the data into information and that can only be meaningful when the 
process analyzes the data against the previously determined goals and objectives. 
Then the next step is to prepare the recommendations based on the results and finally, to 
disseminate the results with recommendations. 

Each evaluation effort has at least one primary audience, the target audience, but 
there are many secondary audiences. Each primary and secondary audience has their 
own perceptions, their own expectations, their own need for the information, and 
probably, they are approaching the information with a different knowledge base. 
Each of the groups identified earlier may hold different stakes or interest* in the 
decisions that are made with the evaluation results. These stakeholder interests 
should come forward at the formulation of the goals and objectives, but do not count on it 
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happening. Stakeholder interests change. If you fail to recognize and deal with these 
interests, there can be devastating results from your evaluation efforts. 

First let us discuss the community political situation because it is very important 
that you understand the political climate that supports the library. There often is a 
difference between who has the authority to make a decision and who really decides 
an issue. Learn whafs hot and what's not. If education is hot in your community, 
emphasize the importance of homework support when evaluating children's services. 
But if education is not hot, if your constituents feel overwhelmed by the resources 
devoted to supporting education, then it is not wise to build an objective and evaluate 
quality or quantity of library services for children around measures that report 
homework support. 

Know what works and what sells. Much of the information we assemble in OEID 
may be used in a budget justification. This ultimate use of the information is always a 
part of my thinking when planning an evaluation. We know that right now the County 
budget office is keen on formulas. The Library has developed a page staffing formula to 
assess the number of pages needed in a library based on several factors including 
circulation, in-library use, building configuration, and collection arrangement. The 
Library is also attempting to create an information services staffing formula based on 
the annual estimate of information questions, quantity of programs and "off-desk" 
responsibilities. In addition, the Library is in the very early stages of developing a 
children's services staifing formula based on the objective that the library will have a 
children's services person available in the building during all hours that community 
and regional libraries are open and that a children's services person will be assigned to 
the information desk during afternoon, evening and weekend hours. The formula will 
also include some demographic characteristics of the service area. In support of 
demand for children's programming, we are recording the number of program 
tuma ways— those children who attempt to sign up for a program and are turned away 
because the registration is full. This data will help measure the unmet need for 
children's programming We do not know if such a formula will successfully document 
the need for additional children's services staff, but it is an example of how we are 



SQtOOL OF U9RAXY AND INTORMATlCN f fXB 



o 

ERIC 



4 



trying to accommodate what we know works now in our County government, our 
supporting community. 

My advice is to recognize poUtic.il reality and try to work within imposed 
constraints. Political realities will have a definite impact on the use of evaluative 
information generated for decision making. The continuing saga about parking at 
Fairfax City Regional Library (FX) is a good example. This library was once the 
system's central facility in the old town center. It still contains the historical 
collection, largely noncirculating, and a special business collection. We know that 95% 
of FX users drive to the library. The library has hopelessly inadequate parking with 
60 parking slots for a regional library that should, according to the facilities plan, 
have at least 150 spots. We have collected abundant information about this critical 
parking problem. I have rented a time clock so I can teU you the average number of 
minutes people stay in the library (20 minutes), and the range of iime people stay (3 
minutes to 3 hours), i can teU you what jurisdiction they come from to use FX. I can tell 
you the percent that park illegally, and I have pages of nasty comments from ang-y 
patrons about the parking situation at that library. I have the information, 
recommendations are clear, but nothing is going to happen until the County and the town 
decide who is going to pay for the parking garage that must be built. Another 
alternative is to remove those special collections and thereby reduce the demand for 
parking. Not a chance. Those materials have always been at that library and will 
stay in spite of all the gathered information that demonstrates the problems facing 
people who want to use the collections. 

Another example of facing community political reality is the story of the small 
branch library that will never close. Every library system probably has one of these 
stories. This 12,000 square foot branch occupies a store front in an apartment complex 
built on a lakeside. FCPL opened a new glitzy regional library about half a mile from 
this tiny branch. The plan was that the small library would close when the new one 
opened. We prepared all kinds of analyses to show cost inefficiencies and low use. 
When word got out that the library would close, the community rose up in a furor. 
Senior citizens live in the apartments around the small branch and community services 
for seniors are hot. I am convinced that this branch will continue to live forever 
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circulating less than 4000 items a month despite all recommendations based on sound 
management information. 

The politics of dealing with library management has been mentioned several times 
during the institute. Most that we do at FCPL is in response to library management's 
need for information. Our role is to support this need. There are several steps in the 
evaluation process whether the idea for the effort originates with you or is brought to 
you. First of all, start with a question that needs an answer. Whenever possible, work 
with the one who is asking the question. Sometimes I do not have that privilege 
because the question comes from the Library Board and I hear it asked through the 
Director. But whenever possible, go to the one who is seeking the information. 
Understand their concerns. Understand if they are or are not willing to act on an answer 
to the question. Do they care enough to act? They must buy into the need for the 
information before you spend your effort collecting and analyzing data. I always ask, 
when someone comes to me with a question, "What will we do differently if we have 
this information?'' 

Once the question has been clarified, identify the pieces of information that are 
needed to answer the question. You must determine tha primary audience for the 
information; it should be the one who asked the question in the first place, but it is not 
always. The secondary audiences should also be considered. Then you must decide if 
these audiences are going to accept the information that you plan to generate as an 
appropriate response to the question* Understand how good the information has to be. 
There is defini. y a level of effort versus quality of data trade-off. You do not want to 
waste time doing more than is needed/ but you certainly want the information good 
enough to be accepted. Find out how soon the information is needed and negotiate a 
realistic timetable. Determine the methodology to be used to collect the data and be 
sure that management approves the amount of effort that is to be spent on collecting 
that data. These costs include expenses for collecting and tabulating data, disruption to 
normal workflow, and staff time consumed by the effort. Minageme ust have a clear 
understanding of these costs. Report the results and recommendations in a timely 
manner. Observe the timetable you negotiated, Then, perhaps the most difficult part, 
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push for discussion of the recommendations and prompt implementation. Do all that 
you can to see that what you have generated does not get buried. 

There are also political considerations concerning your colleagues' role in the effort 
They may not appear to be directly involved with what you are evaluating. It may not 
be clear that they will be affected by the results of your evaluation effort, but often the 
results may recommend a reallocation of resources within the organization fo 
accomplish an objective. This can become a turf issue. I suggest that you keep your 
co-workers informed of all evaluation plans. Encourage other department managers to 
look for information in your study that they can use, expand the ownership. Let them 
share in your effort if possible, but keep the information pieces related. Do not explore 
too much at one time or the effort may be muddled. Be up front with colleagues about 
the disruptions they may experience during the data collection phase. Be very aware 
of any special activities that may be happening in your library during the time that 
you are planning to collect data. Delay your effort if you cannot do anything to resolve 
a potential conflict. It is better to postpone than to collect data and then have to 
apologize for it later. For example, at FCPL we are installing a new circulation system 
during the next fiscal year; I will be very careful about any data collection that 1 ask 
the staff to manage during this busy time. 

Staff have great power over the success of most evaluation efforts. We depend on 
staff to adhere to procedures and collect accurate data. We depend on staff to correctly 
understand the point of the evaluation effort so that they can respond appropriately to 
public queries. It is crucial that we understand, as best we can, staff concerns about an 
evaluation activity before it begins. Evaluation activities are always perceived as 
being done to some person or group unless you have successfully conveyed the concept 
that the activities are conducted with the person or group. This is a very important 
concept. A negative staff reaction to an evaluation can range from outright sabotage to 
simple indifference. Staff attitudes all along that continuum will result in poor data 
and misrepresentations to the public-that can be very dangerous. Ihe best defense 
against this abuse of staff power is communication. Inform staff fully and honestly 
about the purpose of the evaluation. Try to use regular communication channels. Use 
newsletters, and regularly scheduled meetings to inform staff about an upcoming 
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evaluation effort rather than call special meetings. Use these channels unless you 
know they are faulty. If you know there is a poor link in the communication chain, be 
aware and work around it. Try to keep the effort within the normal management 
framework of the library. Brief staff on the evaluation methodology, and explain how 
the results are going to be used Make every effort not to appear to punish them with 
the data they are asked to collect. Report the ults back to staff. 

Staff hostility to an evaluation activity may not be related to the effort at hand. 
If you cannot successfully separate the evaluation activity from what you have 
identified as the cause of the hostility, it may not be the right time to conduct the 
evaluation. Understand the staffs position. It is likely that they are the biggest 
stakeholder in the issue you are trying to evaluate. They may be closest to the activity 
that is under investigation. It is most likely that they are the group that will have 
their work changed as a result of the evaluation. A common reaction from an 
individual or from branch staff is their feeling that a systemwide decision is 
inappropriate for them because their situation is different. They may resist being 
evaluated as a part of the whole. You may encounter this reaction in a competitive 
multi-branch system. You must respond to this attitude or you may not be able to conduct 
the evaluation and gather meaningful results. 

If staff are collecting the data, they must be carefully trained to understand and 
follow the procedures. It is easy to become lax about training if a study is conducted 
regularly. We do not conduct studies or evaluations by memo. With the high turnover 
among FCPL staff, the procedures are always new to some members. When we are 
preparing for a systemwide study, we meet with a survey coordinator assigned by each 
branch We make the training a pleasant event with coffee and cookies. We explain 
the purpose of the evaluation. All procedures are review ed with examples. We discuss 
the preparations needed to ready the branch for the survey such as form numbering and 
sign posting. We offer suggestions for training branch staff. We always recommend that 
branches have a trial run to gear up for the real survey period. If the branch survey 
coordinators are responsible for tabulating results, we carefully present the process. 
After every survey, we ask the branch coordinators to complete a simple, one-page 
evaluation form that asks the following kinds of questions: What would have 
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improved the training session? Were there any unusual incidents in your branch during 
the data collection period that we need to know so that we can better understand the 
results" If a que°tionnaire was used, we ask ii the public experienced any difficulties 
with the form. B.anch managers are also encouraged to add their comments. The 
information from these questionnaires helps us interpret the collected data and may 
reveal staf f objections or mistrust of the process. 

When planning an study, remember to keep procedures as simple and as little 
intrusive as possible. Collect no more data than is needed. Be as careful when 
developing a form that staff will use as you are about the questionnaire you distribute 
to the public. If tally sheets are used, post them at all service points. Just remember 
how many tally sheets are posted so that all will be collected. Change the tally 
sheets often. We do not know why, but staff report that they are more likely to 
remember to record a hash mark if the sheet does not look full and messy. 

Depersonalize the evaluation. Focus on group activities to evaluate unless it is a 
performance appraisal. Identify factors that influence the quality of total service to 
the public, not on the individual action. A good example of that concept was the 
cooperative reference accuracy study conducted in 1984 with Montgomery County (MD) 
and Arlington County (VA) libraries. We wanted to know how accurate were the 
responses given to telephone reference questions and the manner or service 
characteristics demonstrated when the response was given. We wanted to know if 
incorrect information was given and why. What were the causes of incorrect 
information? Was the information out-of-date? Was tne response incomplete or 
totally incorrect? Did the staff person not understand the question? We madv it clear 
from the beginning that we were not reviewing the performance of individuals; we only 
looked at system and branch performance. Branch results were distributed to branch 
managers and were not included in the final study report. We were looking for 
systemwide corrective actions that would improve performance. We did not tie 
recorded transactions to branch schedules and, therefore, maintained the anonymity of 
the staff who handled individual transactions. Another feature of the accuracy study 
involved staff directly in the evaluation process. The study, conducted without special 
funding, utilized information service staff as surrogate patrons. Personnel from one 
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library system called branches of other participating systems. Serving as unobtrusive 
callers, these information services staff became information consumers. Their 
participation as callers heightened their sensitivity about the way they responded to 
a telephone question. 

When considering staff attitudes toward an evaluation, the question of whether or 
not to bring in an objective outsider to manage the process may arise. There are 
trade-offs. Bringing in an outsider may increase the "done unto us" attitude toward the 
effort. Sometimes the expertise to conduct the study may not exist in-house. You may 
realize that bringing in outside talent will add credibility to the results in the eyes of 
the primary audience. These factors must be considered in relationship to the 
individual project. 

As stated earlier it is important to report results and recommendations to staff that 
participated You may expect some objections to the report and these concerns may be 
passed on to the evaluation's primary audience. You may not choose to formally include 
staff objections in the fir.al report, but you can share them informally with 
management. If you can anticipate that the report recommendations will cause 
negative reactions from staff, you may need to explain to them that the results are 
presented to them for their information, not for their approval, and not for their 
endorsement. 

The fifth group to be considered in this discussion is the public, for us that has 
meant library users. It may not be wise to inform the public of an evaluation activity if 
the knowledge might disrupt or change their normal library behavior < ';dch you are 
attempting to study. If the evaluation activity does not intrude on their behavior, 
there is no reason to publicize the effort. If we are distributing a self-administered 
questionnaire, we post a sign on the door saying "Library Survey Today" and on the 
reverse side of the door there is another sign, usually in vivid electric yellow, that says 
''Did you remember to turn in your form?" That is all the publicity that is done* Most 
often you are evaluating normal activity and you want to measure business-as-usual. 
Library users often see a hidden agenda behind a survey. They fear that a survey means 
services are to be curtailed. If you ask them about preferred hours, they assume there is 
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going to be a cutback, maybe even a library closing. If you ask about their use of service, 
they predict the service is going to be changed and certainly not changed to their 
liking. FCPL users often ask staff when given a self- administered questionnaire, 
"What do you want me to say?" Staff response is important. We provide scripts and 
help staff understand that their off-hand remarks can be taken very seriously. Neutral 
responses are always best. FCPL users are very cooperative; public survey response 
rates are often 70 percent or higher. We are very careful about how many times a year 
we approach them with a survey form. 

All survey questionnaires may look alike to a library user. When designing a 
self-administered form, there are several factors to remember. First, provide clear 
instructions on the form itself and give some examples. If you direct them to circle the 
correct answer, draw a circle around the word "circle." If you want an X in a box, 
provide examples. Offer a clear reason why the question is being asked that is accurate 
but fairly neutral. Give advice about where to go if they have questions about the form 
or survey. Be sure to provide a place 'or comments. You will learn much from these 
remarks. Include directions about where to take the form when it is completed. Let the 
respondent be in control of depositing the questionnaire. Do not forget to add a thank 
you at the bottom of the form Insure anonymity in their response? This may be 
difficult in a very small library. Anonymity became a problem when conducting an 
in-library materials use study during a quiet evening in a mini-branch. The library's 
lone patron was dutifully returning the materials he used to the collection boxes when 
he finished with thein. He became concerned when he noticed that the librarian was 
removing the materials and recording them on log sheets. He wanted to know why she 
was checking up on what he was reading. This incident helped us know that this 
method of data collection may not be suitable at all libraries. 

Avoid creating expectations by offering unrealistic options on a survey 
questionnaire. When offering users an opportunity to indicate their preferred service 
hours, offer them choices that can be reasonably fulfilled. If offering a choice of 
services, you pick the options. If you ask open-ended questions about services and hours, 
you may only see answers such as more books and 24 hour-a-day openings. 
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Evaluating children's services adds other concerns. Identifying a methodology to 
collect the data can be particularly challenging when dealing with children At FCTt 
we have used interviewing as a method of including children :n a buildup study and in 
a materials availability study. Parents are under** aiKttWy wary of having thetr 
children interviewed. We train staff interviewers to deal with adults who trot* oc 
answering for the child. If they cannot successfully redirect the quea&x* tor ihr child % 
answer, the adult's responses are recorded, although that interview may not be 
included in the tabulation. If during the interview, the interviewer discover* that the 
child is outside the age range planned for the survey, the interview u mil completed 
Every effort is made to make the child feel that they are a competent and v*lu*d 
respondent. Our experience has shown that children are delightfd 10 haw the adult 
interviewer's attention. A small reward such as a bookmark is also appreciated t» tfcr 
child. Stickers are good rewards because they can idenof> children *!f**dt 
interviewed. 

Whenever possible, pursue evaluation efforts when wu are mo* crru&n tt** th* 
results will be used. Staff energy, time, and goodwill, never mind your own truttr*»M* 
are too valuable to waste. Remember that cost don not equal vahar A cunpfar 
evaluation effort carefully planned, implemented and antiviral ma* nr»uJ< ** r**c&* 
the information that is needed. If you are new at evaluation, start tmil l*a%> * 
simple. Be prepared to learn from your efforts, especially about the pobtv* at **** 
library. Be ready to negotiate throughout the process Remember itmn hfows *c*#% **i 
goals were determined through negotiation Deosnms made with <r **u*+o+ 
recommendations may not ptrsent long-term torr\rr -good hl^vjuows * am pre t e r m % 
resulting actions may not catapult the library all the hih io gpai *********** W 
they may represent a compromise that u accrptaWr u> ali ihr tou\^«n **v? «*t 
the library closer to that moving target 

In conclusion, even though she is not hrrr in mr umt c* fc«* V * :# 
advice: "Kemember that you collect no dau hrtorr >u nanr " 
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"Don't you think we ought to get information about th fectiveness of children's 
librarians in the system before making a change in personnu policy?" 

"Hurrumph!" said Theophilus. "What kind of information? And how are we going 
to get it? People talk about research as though it were fruit growing on a tree and they 
say, Let me go and pick some; if s just what we need; it will do us good. They don't 
realize how difficult it is to do and apply." Theophilus was getting into his stride. 
"And as for library research, there's so little of it and so much of what there is bad or 
ill conceived. Just look at the library literature that is supposed to report it: bitsy, 
superficial, insignificant, irrelevant. Hell's bells!" he exclaimed (eschatological 
exclamations were with it in Nirvnanium), "I've got to act decisively and soon. The 
budget is at a critical stage and the board wants economy and reductions. I can't sit 
around forever waiting for someone to come up with evidence about something. 
Wooliness," he spluttered. "Silliness. Hurrumph!" 

He paused for a moment, looking keenly at Hermione, who had gone rather red 
during this outburst. "I tell you what," he said more gently, "you go away and make an 
appointment to see me one week from now. If you have a good idea and can convince me, 
I'll give you what support I can. But I want results by the end of the year." 

As you can see, Theophilus was a softie really, and he had a very astute notion of 
Hermione' s abilities. 

Later that evening, Hermione described her interview to Nathaniel. 

"Well," he said, "I suppose you were a bit vague, but at least he will give you some 
support." 

Hermione did not reply. She was turning Theophilus' comments over in her mind. 

"The crucial thing, Nat," she said, "is how to convince Theo and others like him 
that children's librarians are needed. That's what I want to prove. I really believe we 
are, but just how are we and our work special? Perhaps one of the most important things 
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is the kind of relationships we develop with kids— maybe in helping them in their 
re .ding. What do you think?" 

Nat had some ideas about this, and the discussion quickly gathered momentum. 
Phrases like reading skills, cognitive development, fantasy play, maturation, ethical 
values, problem solving, and emotional growth were tossed about. 

"That's the children's side," said Nat eventually. "What about us, the children's 
librarians?" Then they discussed the nature of the relationship that seemed to develop 
between the librarian and the children. What encouraged it? What hindered it? How 
was it best categorized and described? 

"What about the library?" said Nat.. 

"Easy," said Hermione. "Collection development, guided reading, summer reading 
programs, dramas/plays, puzzles and game competitions, unstructured access to 
computers, creative use of Sensation Induction. ..." 

"Cost effectiveness, resource allocation, staffing levels, administrative overheads, 
professional development," chimed in Nat. "Ugh! Where to begin?" 

But Hermione was excited. She threw herself wholeheartedly into preparing for 
her interview with Theophilus. The week he had given her flashed by. 

As she once again sat down in his office, she handed him a document boldly labeled 
"Proposal: A Study of the Role and Effectiveness of Children's Library Service in the 
Greensward Tri-Urban Conglomerate System (GTUCS)." 

Theophilus glanced at % riffled through the pages, and then placed it on the desk 
in front of him. 

"Later for that," he said. "You tell me in your own words what you wcjtf to do." 



Rtywarxi SCHOOL OF LIBRARY AND INFORMATION STUDIES 

292 

ERIC 25.;. 



So Hermione told him. She spoke simply and clearly, trying to keep in mind what 
his interests were. At one point she referred to the proposal. There were certain 
technical F°blems in the research methodology she wanted to use and she hoped to get 
expert help from Nirvnanium Central University, the main campus of which was 
located in Cogitatibus. "I spell out the problem and the reasons for getting in a 
consultant in the proposal/' she said, "but 1 assume you'd rather I didn't go into them 
now." Her voice rose in question. 

"Certainly, certainly, hurrumph! Later!" said Theophilus. 

At the end of five minutes, Hermione thought she had said enough and she rapidly 
listed on her fingers the kinds of support she hoped Theophilus would give her. 

A brief silence ensued. She was a shrewd observer, perhaps shrewder than 
Theophilvs, and she could tell he was impressed. But he was noncommittal when he 
did speak. 

"I'll let you know vhat I think when I have read the paper." He gestured at the 
proposal and then stooo up. "Back to work. Back to work." He made shooing gestures 
with his hands. "There's little time and much to be done. Can't have people lounging 
about all day thinking, can we? No. No. Hurrumph!" 

Hermione did not know what to make of this. She feared the worst and began to 
feel very depressed. Nat took her out to dinner to cheer her up, but she was tense and a 
bit quarrelsome. They went to bed early. 

Next morning, promptly at nine o'dock, Hermione was called to the videophone. It 
was Thaophilus. She felt a pang of excitement and, because her hands had started to 
tremble, she faced the screen holding them behind her back. She smiled uncertainly at 
Theophilus. 

"Well, young lady," he said briskly. "About that proposal of yours: you can go 
ahead with your research. All I can do is let you have some time off during working 
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hours, but you'll have to clear actual times with your supervisor. There's no money for a 
consultant. You talk to the people at Nirvnanium Central U.; maybe they'll help 
anyway." He reached over his desk to the videophone switch, but paused suddenly. 
"Oh! I'll want your report before the next full assembly of the Board of Library 
Governors." His image flickered out. 

Hermione was left breathless by this. The full assembly of the Board of Library 
Governors was barely six months away. She wasn't even going to have until the end of 
the year. 

"I can do it/' she said confidently to Nat later. "At least I can organize it. If we are 
to get solid results in that time, I'm going to need lots of help. How to get it is the 
problem. There are the others of us in the system. There are the branch heads and the 
central administrative staff. All will be involved to some extent." 

Nat had some opinions about this from observing whrt had happened during the 
preceding week. "You have to involve the various groups from the beginning," he said. 
"They've got to have input. You've got to get them curious about what you're doing and 
what you might find. The planning stage of research like you want to do," he observed 
rather sententiously, "is actually a kind of preliminary reporting on it: writing and 
talking to different people with different interests so that they know enough to be 
sympathetic and cooperative and..." he fumbled for a word, "ah...unthreatened." 

And in the next few weeks she reported in the mo*t diverse kinds of ways on what 
she wanted to do and on the initial stages of her work. She spoke to children's 
librarians in the system, individually and in groups. 5he made a point of seeking out 
influential branch heads (especially her own) and briefing them. She wrote a number of 
< ' mos. Some of her colleagues were eager to help; some were indifferent; some were 
covertly hostile. At Nirvnanium Central U. she began to meet weekly with a friendly 
statistician and two of the library school faculty. She read the literature. She 
developed questionnaires and interview schedules and special tests. She drew samples 
and learned how to code data sheets. 
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And as the time flew by, she kept various groups informed of developments. She 
addressed a staff meeting on the growth of children's services in the past decade. She 
drew up a fortnightly report for her branch head. It was a model of succinctness in its 
indication of what time she needed, why, and where it would be spent. On her own 
initiative she made an appointment to see Theophilus to give him a progress report. 

And as the time flew by, sometimes she felt elation, sometimes despair. Sometimes 
she was so far down in the dumps she felt completely incompetent and unable to cany 
her work through to the end. Other times she felt confident, energetic, and marvelously 
resourceful. Sometimes she suspected that some of her colleagues in the system were 
sneering at her as pretentious and impractical--and sometimes they were and she was. 
She frequenUy snapped at Nathaniel. Once or twice, it seemed deliberately, she 
provoked a violent quarrel with him Withdrawn and preoccupied, she could not stand 
being in the kitchen. She burned things, dropped things, misplaced things, cut her 
fingers. Nathaniel began to wish that she had never caught the bug for research. 
Eventually he couldn't stand it any longer and took over all the household chores that 
they had previously shared equally— though she was the better cook. 

And the time flew by until at last it was only a week from the full assembly of the 
Board of Library Governors. And then Hermione made her first mistake. 

Working until the early hours of the morning several days running, Hermione 
prepared a report on her work. It grew larger and larger and more and more complex. 
She described every stage of her thinking, every twist and turning in the development 
of her research. She explained why certain techniques were tried and abandoned; she 
discussed false leads, empty clues, disappointed hopes. The methodological section 
became a chronicle of what she had learned of statistics. She prepared graphs and 
figures and tables and charts. She spoke about variance and significance and levels of 
confidence and one- and two-tailed t-tests and partialing out zero-order correlations 
and rotating orthogonal axes. She justified, speculated, dilated, expatiated. And when 
she handed her thick, heavy report, which had been hastily copied and assembled, to 
Theophilus, she felt that she had accounted fully and completely for all that she had 
done. Not to put too fine a point on it, she was smug. 
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Theophilus, however, was exasperated. "I can't possibly read and assimilate this 
before the board meeting," he said, hefting the report in his hands. He turned the 
pages rapidly for a few minutes pausing here and there to read a heading, a sentence, 
the caption to a figure. 'The summary list of recommendations and conclusions— where 
is it?" he demanded. "Where is it?" 

Hermione admitted there wasn't one. 

"Where is figure 6?" he asked. '1 see 5 and 7 but no 6." 

Hermione looked over his shoulder. "Oh, 7 is misnumbered," she said. "It's really 6. 
But it doesn't matter, that figure is not important." 

"There is no way to present this to the board as it is," concluded Theophilus. He 
shook his head. "A bad business. A bad business. Hurrumph!" He thought for a moment. 
"You can make the report. I shall make a short time available on the agenda for you. 
There is interest in what you have been doing; everyone is aware of it. So you tell them 
yourself/' He picked up the report and waved it at her. 'Too long. Too long. Hurrumph." 

And then Hermione made her second mistake. 

The full assembly of the Board of Library Governors was very large and very 
august. There were thirty-five members from all the highest walks of life in 
Nirvnanium. The president was a hereditary governor, Sri Bombasticus Patheticum, a 
tall, thin man whom no one much liked. But he was an excellent chairman: he was mean 
and he wasted no time. Many were the garrulous and unprepared who had quailed and 
fallen silent before his inimical stare. 

Hermione was elated at the idea of presenting her work to the board. She had done 
so much, come so far, she was sure she could convince them of the importance of her 
findings. Hipping through her report, she concluded that four tables and two figures 
were central and she decided she would present them to the board. There was no time to 



Riywird 



SCHOOL OF LIBRARY AND INFORMATION STUDIES 



have them transformed into multicolor 3-D screen format (the technicians needed two 
weeks' notice), so she decided to use an old-fashioned overhead projector and a heat 
transparency. 

At the end of the second day of the board's meeting, the agenda item "Other 
Business" was reached. Theophilus respectfully indicated to the president that there 
were ten items to be dealt with under this heading if possible. Item number 7 was "A 
Report by Hermione Scientiarum on Children's Services in GTUCS." 

Hermione went to the microphone, switched on the overhead projector, placed her 
first transparencies face down beside her, and began her report. First at some length she 
thanked all those who had helped her. She spoke softly, standing back from the 
microphone. 

"Speak up. Speak up," said the president testily. She did but nothing much 
happened. 

"Lean into the microphone," he snapped. "Bend it toward you." 

Adjusting the microphone, Hermione began to talk about general problems of 
children's library service and how shr became interested in them. 

"Get to the point. Get to the point," interrupted the president, his rather 
protuberant eyes staring at her. This flustered Hermione and she decided to put on the 
first of her transparencies. As she announced what the transparency represented, she 
leaned forward to place it on the machine and her voice faded away so no one heard 
what it was. When she looked up she found the image was out of focus and half of it 
was off the screen. 

"Excuse me," she said, and started to fiddle with the focus. Then she pushed the 
machine sideways but the image was also below the screen so she had to screw the 
movable feet to adjust the elevation. One or two of the governors peered at the screen, 
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pointing to the parts of the image. There were one or two loud mutters, "What is it? 
What is it?" and a slight rumble of talk began. 

Hermione plunged into an explanation of the figures on the screen, but they were too 
small to read. 

\ "Can't you magnify the image?" demanded the president. Hermione tried but not 

very successfully. 

"Let me go on to the next," she said. 'This is a figure that show the distribution of 
children's ages, the frequency of their interactions with the librarians, and adjusted 
figures for their volume of reading." 

Very quickly, one after another, Hermione projected the transparencies, discussing 
them in considerable technical detail. In every case the figures and captions were small 
and most of the images were faint. The buzz of incidental conversation gradually 
increased. Eventually Sri Bombasticus knocked his gavel on the table. 

"Thank you, Miss Scientiarum/' he said. "I think we must leave your— ah— -very 
interesting report there. Perhaps a copy could be left with the board secretary should 
members of the board wish to examine it. The next item of business, please" 

Hermione gasped and sat down. She had not presented her conclusions. She looked 
at her watch. She he spoken for fifteen minutes. It was all over. She felt tears 
pricking at her eyelids. 

After the meeting Theophilus came up to her. "That report of yours was no good," 
he said in his blunt way. "No good at all. Hurrumph! You didn't think about your 
audience and what their interests would be or how they would be feeling— tired out. 
heavens above, they've been meeting for nearly two days solid! I'm going to assume 
that you really did find out interesting things that we should know about. At the 
moment I'm acting on faith. In two weeks the board's executive committee will be 
meeting. I want distributable copies of the report and for you to give a fifteen-minute 
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talk on it. If you agree to this/' he poked his ringer at her, "You had better learn from 
today's disaster...," a strong word even for Theophilus. 



Of course Hermione could do it and of course she could learn from her mistakes. She 
worked out the strategy with Nat. The verbal and the written reports would be 
complementary. She would cut down the written report by half. All the statistics and 
technical detail would be moved to an appendix. A clearly labeled table of contents 
would follow the tide page. A summary, two pages in length, wo.-Ud follow that. Her 
language would be simple ard clear. She would indicate what the problem was that 
she was investigating, why it was of interest, and what range of consequences might 
follow from studying it. And then she would tell them what she found and what she 
thought that meant for library services in the system. She would use no jargon. She 
would write in such a way that she did not intrude as a person in her account of the 
subject, so that what she was describing was not what she did but rather what was 
discovered and how the discoveries were made. 

Her talk, on the other hand, would be more personal and informal. She would net 
use any props except a simple table showing percentages and she would make copies of 
this for each committee member. 

She tried out her talk on Nat and he made some suggestions as to how to capture the 
interest of the committee. Indeed, he suggested that she start out with a major finding 
first, couched in the form "Did you know...?" This would be dramatic and arresting. 

Hermione was understandably nervous when she rose to speak before the executive 
committee. Sri Bombasticus was there, looking unfriendly. Theophilus was frowning. 
But all of her work of preparation was justified. She did not quite fill her fifteen 
minutes. The handout was distributed without any incident (she had fantasized that 
just at that point she would fall over, or Sri Bombasticus would look at her fiercely and 
she would drop her notes). There was a lively discussion in which Sri Bombasticus 
joined and one of the members of the committee complimented her on her work, saying 
he would take her report away and read it with great interest. 
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Theophilus was delighted. "Knew you could do it," he said gruffly afterwards. 
"Knew it all along. We'll issue a press release. I think a number of other libraries will 
be interested in what you have found, Draft one for me, please." 

This was not an ending but a beginning. The press release led to a notice appearing 
in the Nirvnanium National Library Journal As a result, a number of ^quests for 
Hermione's report were received. She was asked to give a talk at the library school at 
Nirvnanium Central University on the methodologies she had employed in her study. 
The research forum of the Nirvnanium Library Association asked her to report her 
findings at its annual meeting. She had a letter from the editor of the Nirvnanium 
Library Quarterly, a most prestigious and scholarly journal. He suggested she might 
like to write up her report as an article so that her contribution could become formally a 
part of the permanent record of knowledge in the field. She found the prospect of this a 
little alarming but Nathaniel encouraged her. He drew her attention to an article that 
had appeared in May 1980 in a foreign journal from another time called College & 
Research Libraries. The article was entitled "Publishing Library Research" and was 
written by W. Boyd Rayward— a very peculiar name to Nirvnanians like Hermione, 
Theophilus, and Nathaniel. The article suggested how one might go about publishing 
different kinds of reports for different kinds of audiences in a range of different kinds of 
journals. She tried to write an article for the Quarterly but it was rejected. 

During all this time Hermione was nagged by the realization that there were a 
number of unanswered questions in what she had done, So she proposed to Theophilus 
that really she needed to do more research. He was interested but suggested she should 
get a grant from the National Library Authority to pursue any further research. 

And this was another beginning. Hermione gradually devoted more and more time 
to research. As a result she also spent much more time writing and talking about her 
work. There were informal gatherings at which she spoke in Cogitatibus, and formal 
professional meetings of a regional and national kind. Some of her articles were 
popular and some were scholarly. It did not take her long to begin to publish important 
articles in the Nirvnanium Library Quarterly. And gradually years began to pass and 
what had been articles became books. Eventually Hermione was recognized as an 
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authority in her field; she was highly respected, she was frequently consulted, she was 
even appointed to the faculty of the library school. But she didn't like to teach and so 
she took a special consultant's job with the Nirvnanium National Library Authority. 

And then one morning she received a letter that looked very official in a thick, 
creamy envelope embossed with the seal of Nirvnanium Central University. The letter 
was from the president of the university and informed her that she had been selected to 
be awarded an honorary doctor of humane letters degree in recognition of her 
contributions to library research. This was a great honor. Hermione looked forward to 
the ceremony with considerable excitement. And so did Nathaniel and Heliopola and 
Arkarnia. Heliopola and Arkarnia were the twin daughters of Nathaniel and 
Hermione, who had decided some years before to get married and have children. 

At last the summer convocation of Nirvnanium Central University arrived. On the 
platform with Hermione and the other candidates for honorary degrees were a number 
of great and famous figures in Nirvnanium's society, including the now very frail and 
elderly Sri Bombasticus Patheticum, who no longer seemed as fierce to Hermione as he 
had all those many years ago. 

Eventually her turn came. Her presenter, the statistician with whom she had 
worked on her first research project and who was now head of the department of 
statistics in the university, called out her name. She stood before the president while 
her presenter read out her citation. In sonorous phrases it chronicled the major events in 
her career, recalled some of her books, and some of her many professional honors. 

Then the citation went on: "Above all, you have shown consummate skill in 
reporting your research. You have always written and spoken with a clear sense of 
purpose and of audience. Consequently the humblest librarian and the most learned 
library scholar have been able to respond to what you have said and written. Your style 
is simple, direct, clear, unforced. Your tone has been a judicious balance of the personal 
in speaking and the formal in writing. You have eschewed the use of jargon and always 
sought in the presentation of your ideas a rigorous logical development. Thus you have 
been able successfully and accurately to inform a wide range of professional and lay 



Riywtrd 

UNIVERSITY OF WISCONSIN— MADISON 

301 



2:#7 



persons about your work in such a way as to influence practice and to add significantly to 
the cumulation of knowledge in the field. Madam President, I have the honor to present 
to you Hermione Scientiarum as a candidate for the degree of doctor of humane letters 
honoris causa." 

In the audience Nathaniel glowed with pride, though Heliopola and Arkarnia 
had begun to wiggle, for the session was a long one. Elsewhere in the audience, unknown 
to Hermione and her family, sat another figure from her past who, white-haired and 
bent, leaned on a cane. This was Theophilus Grouchmore, whose rheumy old eyes 
became even redder and more moist as Hermione stood before the president. He said 
softly, clearing his throat, "Hurrumph! Hurrumph!" 

Hermione Scientiarum lived to a great age. When she died, she was buried beneath 
a tombstone that read, simply, Library Researcher. 

This is not the end of my story either, because Hermione immediately went to the 
librarian's heaven. Saint Jerome had been deputed by Saint Peter to guard the gate to 
this particular part of paradise. He greeted Hermione warmly when she presented 
herself to him and swung the gate open. Behold! Massed as far as the eye could see was 
a host of librarians chanting in welcome: "Crescat scientia vita excolatur."* 

This is the end of my story. 



•"Let knowledge grow from more to more and so be human life enriched." This just happens to be 
the motto of the University of Chicago also. 
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